-- Logs begin at Fri 2022-09-02 08:41:20 UTC, end at Fri 2022-09-02 10:30:28 UTC. -- Sep 02 09:12:44.942977 dv-cinder-850831-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 02 09:12:44.953756 dv-cinder-850831-3-iscsi devstack@keystone.service[81599]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 02 09:12:44.956231 dv-cinder-850831-3-iscsi devstack@keystone.service[81599]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Sep 2 09:12:44 2022] *** Sep 02 09:12:44.956231 dv-cinder-850831-3-iscsi devstack@keystone.service[81599]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 02 09:12:44.956231 dv-cinder-850831-3-iscsi devstack@keystone.service[81599]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 02 09:12:44.956231 dv-cinder-850831-3-iscsi devstack@keystone.service[81599]: nodename: dv-cinder-850831-3-iscsi Sep 02 09:12:44.956231 dv-cinder-850831-3-iscsi devstack@keystone.service[81599]: machine: x86_64 Sep 02 09:12:44.956231 dv-cinder-850831-3-iscsi devstack@keystone.service[81599]: clock source: unix Sep 02 09:12:44.956231 dv-cinder-850831-3-iscsi devstack@keystone.service[81599]: pcre jit disabled Sep 02 09:12:44.956231 dv-cinder-850831-3-iscsi devstack@keystone.service[81599]: detected number of CPU cores: 4 Sep 02 09:12:44.956231 dv-cinder-850831-3-iscsi devstack@keystone.service[81599]: current working directory: / Sep 02 09:12:44.956231 dv-cinder-850831-3-iscsi devstack@keystone.service[81599]: detected binary path: /usr/bin/uwsgi-core Sep 02 09:12:44.956231 dv-cinder-850831-3-iscsi devstack@keystone.service[81599]: your processes number limit is 47680 Sep 02 09:12:44.956231 dv-cinder-850831-3-iscsi devstack@keystone.service[81599]: your memory page size is 4096 bytes Sep 02 09:12:44.956231 dv-cinder-850831-3-iscsi devstack@keystone.service[81599]: detected max file descriptor number: 2048 Sep 02 09:12:44.956231 dv-cinder-850831-3-iscsi devstack@keystone.service[81599]: lock engine: pthread robust mutexes Sep 02 09:12:44.961399 dv-cinder-850831-3-iscsi devstack@keystone.service[81599]: thunder lock: enabled Sep 02 09:12:44.961399 dv-cinder-850831-3-iscsi devstack@keystone.service[81599]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 02 09:12:44.961399 dv-cinder-850831-3-iscsi devstack@keystone.service[81599]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 02 09:12:44.994821 dv-cinder-850831-3-iscsi devstack@keystone.service[81599]: Python main interpreter initialized at 0x55be59e299c0 Sep 02 09:12:44.994821 dv-cinder-850831-3-iscsi devstack@keystone.service[81599]: python threads support enabled Sep 02 09:12:44.994821 dv-cinder-850831-3-iscsi devstack@keystone.service[81599]: your server socket listen backlog is limited to 100 connections Sep 02 09:12:44.994821 dv-cinder-850831-3-iscsi devstack@keystone.service[81599]: your mercy for graceful operations on workers is 90 seconds Sep 02 09:12:44.995150 dv-cinder-850831-3-iscsi devstack@keystone.service[81599]: mapped 403077 bytes (393 KB) for 2 cores Sep 02 09:12:44.995333 dv-cinder-850831-3-iscsi devstack@keystone.service[81599]: *** Operational MODE: preforking *** Sep 02 09:12:44.995608 dv-cinder-850831-3-iscsi devstack@keystone.service[81599]: *** uWSGI is running in multiple interpreter mode *** Sep 02 09:12:44.995608 dv-cinder-850831-3-iscsi devstack@keystone.service[81599]: spawned uWSGI master process (pid: 81599) Sep 02 09:12:44.995796 dv-cinder-850831-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 02 09:12:44.996595 dv-cinder-850831-3-iscsi devstack@keystone.service[81599]: spawned uWSGI worker 1 (pid: 81611, cores: 1) Sep 02 09:12:44.996686 dv-cinder-850831-3-iscsi devstack@keystone.service[81599]: spawned uWSGI worker 2 (pid: 81612, cores: 1) Sep 02 09:12:44.998578 dv-cinder-850831-3-iscsi devstack@keystone.service[81599]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 02 09:12:49.331924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 02 09:12:49.331924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: from cryptography.utils import int_from_bytes Sep 02 09:12:49.332669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 02 09:12:49.332669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: from cryptography.utils import int_from_bytes Sep 02 09:12:49.333073 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 02 09:12:49.333073 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: from cryptography.utils import int_from_bytes Sep 02 09:12:49.333179 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 02 09:12:49.333179 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: from cryptography.utils import int_from_bytes Sep 02 09:12:49.382178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 02 09:12:49.382354 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 02 09:12:49.382543 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] command line args: [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 02 09:12:49.382727 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 02 09:12:49.382842 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 02 09:12:49.382921 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 02 09:12:49.383020 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ================================================================================ {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 02 09:12:49.383097 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] command line args: [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 02 09:12:49.383271 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] admin_token = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.383354 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 02 09:12:49.383516 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] config_dir = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.383600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ================================================================================ {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 02 09:12:49.383703 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.383786 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] admin_token = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.383882 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] config_source = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.384011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] config_dir = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.384092 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] debug = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.384201 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.384288 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.384492 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] config_source = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.384619 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] debug = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.384703 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.384774 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.384838 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.385008 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.385121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.385224 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.385329 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.385431 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.385527 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] list_limit = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.385631 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] log_config_append = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.385732 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.385826 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.385925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.386071 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] log_dir = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.386175 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] log_config_append = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.386276 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] log_file = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.386371 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.386477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.386605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] log_dir = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.386605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] log_file = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.386826 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.386908 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.387029 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.387104 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.387224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.387224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.387462 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.388081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.388162 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.388287 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.388410 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.388531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.388652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.388772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.388886 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.389005 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.389126 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.389241 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.389359 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.389494 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.389635 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.389743 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.389848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.389968 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.390095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.390198 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.390311 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.390311 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.390541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.390661 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.390792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] publish_errors = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.390918 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.391097 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.57/identity {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.391207 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.391325 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] publish_errors = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.391499 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.391615 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.391734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.391853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.391966 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.392086 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.392207 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] standard_threads = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.392327 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.392440 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.392559 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.392677 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.392802 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] standard_threads = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.392922 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.392922 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] use_journal = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.393150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.393269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] use_json = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.393388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.393512 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] use_stderr = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.393632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.393746 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] use_syslog = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.393858 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] use_journal = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.393971 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.394085 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] use_json = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.394085 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] use_stderr = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.394306 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.394425 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] use_syslog = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.394538 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.394650 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 02 09:12:49.394782 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.394972 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.395089 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.395200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.395300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.395394 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.395494 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.395599 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.395701 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.395801 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.395896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.395989 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.396088 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.396238 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.396335 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.396429 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.396528 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.396635 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.396742 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.396742 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.396936 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.397036 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.397137 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.397237 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.397342 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.397451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.397569 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.397646 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.397724 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.397724 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.397859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.397936 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.398006 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.398075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.398150 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.398305 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.398392 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.398562 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.398639 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.398835 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.398914 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.399077 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.399167 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.399336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.399421 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.399567 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.399644 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.399799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.399957 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.400037 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.400108 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.400256 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.400339 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.400544 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.400638 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.400787 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.400865 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.401015 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.401092 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.401245 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.401332 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.401489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.401566 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.401720 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.401796 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.401954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.402038 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.402197 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.402279 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.402428 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.402547 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.402661 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.402806 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.402921 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.403091 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.403287 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.403439 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.403539 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.403642 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.403738 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.403832 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.403926 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.404050 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.404160 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.404257 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.404357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.404452 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.404568 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.404682 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.404816 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.404932 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.405045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.405146 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.405263 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.405383 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.405494 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.405616 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.405729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.405854 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.405979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.406083 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.406233 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.406357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.406505 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.406697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.406849 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.406959 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.406959 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.407184 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.407296 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.407409 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.407523 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.407636 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.407750 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.407876 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.407981 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.408093 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.408206 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.408325 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.408440 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.408565 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.408669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.408791 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.408912 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.409026 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.409140 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.409260 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.409260 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.409484 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.409597 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.409711 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.409711 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.409915 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.410060 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.410175 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.410289 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.410402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.410514 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.410627 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.410766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.410885 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.410885 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.411106 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.411265 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.411382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.411566 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.411683 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.411797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.411912 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.412025 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.412137 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.412258 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.412372 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.412486 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.412598 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.412711 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.412824 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.412949 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.413076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.413191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.413305 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.413419 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.413532 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.413645 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.413767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.413881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.413996 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.414115 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.414212 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.414307 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.414409 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.414511 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.414608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.414769 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.419451 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.419576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.419665 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.419739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.419816 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.419888 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.419958 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.420028 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.420106 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.420184 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.420256 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.420332 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.420403 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.420473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.420557 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.420630 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.420701 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.420771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.420841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.420911 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.420988 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.421060 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.421130 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.421200 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.421270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.421345 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.421424 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.421496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.421566 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.421636 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.421706 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.421775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.421852 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.421924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.421995 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.422065 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] auth.external = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.422135 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.422234 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.422314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.422387 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.422466 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.422539 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.422610 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.external = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.422689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] auth.password = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.422805 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.422877 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] auth.token = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.422877 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.423019 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.423090 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.423161 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.423231 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.423340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.password = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.423431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.423555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] auth.token = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.423631 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.423703 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.423773 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.423845 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.423924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.424036 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.424127 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] credential.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.424202 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.424273 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.424344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.424423 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.424765 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.424844 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.424916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.424987 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.425067 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.425146 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.425218 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.425289 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.425365 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.425438 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.425517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.425595 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.425723 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.425800 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.425872 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.425944 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.426022 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.426094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.426165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.426281 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.426396 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.426476 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.426627 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.426722 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.426798 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.426870 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.426941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.427011 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.427094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.427168 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.427239 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.427309 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.427379 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.427450 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.427450 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.427592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.427592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.427721 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.427798 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.427870 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.427947 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.428019 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.428091 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.428160 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.428230 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.428300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.428377 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.428450 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.428520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.428590 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.428660 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.428731 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.428808 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.428880 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.428958 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.429030 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.429100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.429170 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.429248 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.429344 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.429468 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.429565 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.429644 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.429705 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.429783 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.429857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.429928 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.429999 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.430116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.430192 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.430316 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.430398 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.430548 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.430627 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.430804 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.430886 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.431048 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.431129 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.431292 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.431370 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.431527 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.431604 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.431764 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.431844 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.432000 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.432079 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.432248 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.432334 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.432485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.432563 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.432720 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.432798 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.432947 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.433035 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.433182 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.433261 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.433410 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.433488 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.433642 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.433728 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.433884 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.433970 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.434122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.434200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.434900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.437635 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.437720 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.437793 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.437864 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.437934 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.438005 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.438089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.438162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.438270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.438349 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.438427 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.438499 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.438577 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.438650 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.438755 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.438830 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.438913 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.438987 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.439066 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.439138 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.439210 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.439280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.439350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.439421 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.439499 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.439499 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.439629 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.439700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.439776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.439848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.439926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.439998 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.440068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.440139 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.440210 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.440281 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.440359 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.440448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.440592 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.440670 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.440742 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.440813 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.440892 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.440964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.441035 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.441105 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.441175 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.441247 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.441326 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.441399 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.441470 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.441541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.441611 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.441682 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.441837 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.441918 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.441990 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.442061 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.442131 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.442201 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.442279 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.442351 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.442422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.442493 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.442563 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.442634 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.442731 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.442859 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.442939 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.443112 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.443190 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.443351 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.443439 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.443586 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.443665 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.443826 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.443905 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.444065 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.444151 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.444304 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.444383 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.444539 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.444618 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.444767 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.444846 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.445014 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.445098 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.445253 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.445331 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.445488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.445568 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.445724 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.445802 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.445956 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.446033 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.446195 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.446274 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.446451 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.446589 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.446749 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.446840 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.446960 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.447041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.447198 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.447276 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.447439 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.447517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.447659 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.447735 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.447897 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.447975 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.448130 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.448208 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.448376 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.448455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.448615 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.448694 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.448846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.448924 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.449092 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.449215 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.449333 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.449414 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.449585 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.449666 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.449843 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.450527 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.450606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.450678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.450874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.450963 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.451122 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.451204 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.451376 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.451455 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.451629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.451708 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.451889 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.451992 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.452140 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.452216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.452371 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] role.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.452459 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.452613 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.452693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] role.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.452841 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.452920 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.453088 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.453166 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.453332 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.453412 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.453570 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] role.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.453648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.453814 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.453893 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] role.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.454046 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.454126 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.454292 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.454629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.454828 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.454910 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.455080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.455169 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.455314 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.455393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.455560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.455639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.455805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.455885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.456050 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.456129 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.456293 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.456380 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.456528 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.456693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.456772 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.456931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.457011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.457180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.457261 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.457429 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.457508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.457676 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.457921 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.458015 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.458152 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.458231 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.458402 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.458481 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.458650 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.460636 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.460718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.460780 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.460839 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.460898 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.460957 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.461025 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.461087 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.461153 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.461153 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.461258 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.461316 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.461382 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.461475 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.461588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.461740 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.461845 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.461913 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.461913 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.462030 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.462030 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.462141 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.462202 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.462299 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.462299 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.462464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.462524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.462583 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.462641 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.462698 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.462806 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.462867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.462926 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.462983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.463040 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.463097 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.463163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.463507 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.463624 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.463797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.463911 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.463911 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.464129 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.464209 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.464282 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.464353 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.464425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.464568 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.464705 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.464790 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.464901 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.465035 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.465176 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.465264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.465408 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.465531 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.465693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.465818 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.465958 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.466038 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.466218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.connection = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.466335 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.466462 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.466567 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.466737 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.466859 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.connection = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.466968 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.467106 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.467197 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.467358 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.467437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.467598 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.467677 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.467842 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.467999 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.468076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.468149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.468307 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.468386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.468547 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.468635 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.468788 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.468866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.469023 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.469169 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.469271 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.469378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.469512 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.469596 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.469750 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.469829 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.469977 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.470064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.470215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.470295 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.470449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.470529 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.470602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 02 09:12:49.470684 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.470942 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.471181 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 02 09:12:49.471353 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81611) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 02 09:12:49.500596 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.500822 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.500926 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.501046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.501133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.501314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.509354 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.509966 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.538030 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.538254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.538370 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.538602 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.605342 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.605491 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.657224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.657385 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.657458 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.657558 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.820377 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.820599 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.820823 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.821045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.821261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.821476 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.823596 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.823828 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.824052 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.824273 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.824492 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.824709 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.846908 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.847021 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.847180 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.847308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.847395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.847492 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.847616 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.847701 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.847836 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.847921 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.848052 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.848148 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.848267 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.848360 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.860037 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.860349 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.873358 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.873588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.873691 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.873800 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.873875 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.874025 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.874109 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.874249 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.874341 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.874462 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.874539 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.874679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.874778 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.883939 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81611) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 02 09:12:49.899536 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.899748 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 02 09:12:49.900047 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.900255 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 02 09:12:49.900353 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.900546 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 02 09:12:49.900630 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.900780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 02 09:12:49.900863 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.901052 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.901145 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.901313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.901395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.901571 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.901653 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.901824 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.911094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.911378 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.911525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.911656 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.911806 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.911947 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.912078 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.912361 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.919492 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:49.919923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:49.922448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:49.922759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.922844 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:49.923051 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.923134 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.923340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.923424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.923632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.923716 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.923900 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.923982 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.924156 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.924287 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.924408 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.924569 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.924681 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.924849 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.924932 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.925132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.925213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.925408 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.925497 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.948751 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.948856 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.950230 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:49.950375 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:49.950514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.950664 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.955576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:49.955674 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:49.955858 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.955942 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.956142 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.956228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.962495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:49.962691 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:49.962789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.963006 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.967423 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:49.967919 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.968401 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.968820 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:49.968905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.969120 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.969379 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.969462 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.969684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.969884 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.969976 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.970216 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.970360 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.970483 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.970775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.970879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.987675 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.987958 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.989484 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.989779 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.992187 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:49.993818 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:49.995452 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.995770 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.997562 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:49.997861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.000686 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:50.002129 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:50.003814 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:50.005472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:50.007049 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.007350 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.007637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.007925 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.008213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.008491 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.008677 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.008985 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.009277 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.009568 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.009862 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.010145 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.024519 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:50.024806 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.025075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.029782 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.029872 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:50.029872 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.029872 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.029872 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.032799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.034310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.034509 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.035912 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.036075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.036193 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.037643 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.037927 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.038837 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:50.039123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.039416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.039701 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.040099 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.040735 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:50.041026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.041342 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.041631 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.041924 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.051965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:50.052251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.052534 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.052812 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.058449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.058449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.058449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.058721 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:50.058721 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.058721 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.058721 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.058721 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.058721 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.059089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.069246 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:50.071173 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:50.072237 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:50.074100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:50.075245 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.076685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.077093 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.078155 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:50.078450 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.078657 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.078881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.080441 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:50.080735 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.081040 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.086922 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.087569 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.088388 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:50.089173 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:50.091800 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.092200 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.092488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.092666 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.092752 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.092974 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.093266 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.093550 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.100601 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:50.100892 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.101175 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.101602 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:50.101900 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.102187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.108708 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:50.115828 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.115828 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.115828 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.116036 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81611) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 02 09:12:50.116109 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.116185 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.116260 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.116341 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.116417 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.116417 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.116561 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.116646 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.116722 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.116797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.116906 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.117005 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.117082 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.117157 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.117231 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.117306 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.117391 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.117468 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.117543 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.117543 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.117543 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.117543 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.117543 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81611) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 02 09:12:50.146185 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55be59e299c0 pid: 81612 (default app) Sep 02 09:12:50.147952 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55be59e299c0 pid: 81611 (default app) Sep 02 09:12:50.149662 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acfa5b-5f77-42ce-8f9d-baaeb339f2dc None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:50.149994 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acfa5b-5f77-42ce-8f9d-baaeb339f2dc None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:50.150311 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acfa5b-5f77-42ce-8f9d-baaeb339f2dc None None] PATH_INFO: `/v3` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:50.155075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1/1] 10.222.0.57 () {50 vars in 791 bytes} [Fri Sep 2 09:12:50 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:12:57.669227 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854605a7-cec2-41e3-890d-f8fcdd3897ba None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:57.669740 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854605a7-cec2-41e3-890d-f8fcdd3897ba None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:57.669826 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854605a7-cec2-41e3-890d-f8fcdd3897ba None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:57.674294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1/2] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:12:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:12:57.679439 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450b2c2-b8e8-4b22-9be7-ed5bafd9fdbc None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:57.679884 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450b2c2-b8e8-4b22-9be7-ed5bafd9fdbc None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:57.679979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0450b2c2-b8e8-4b22-9be7-ed5bafd9fdbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:57.751793 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: INFO dbcounter [None req-0450b2c2-b8e8-4b22-9be7-ed5bafd9fdbc None None] Registered counter for database keystone Sep 02 09:12:57.773223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG oslo_db.sqlalchemy.engines [None req-0450b2c2-b8e8-4b22-9be7-ed5bafd9fdbc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81612) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 02 09:12:57.779497 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writer thread running {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 02 09:12:57.978139 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG passlib.handlers.bcrypt [None req-0450b2c2-b8e8-4b22-9be7-ed5bafd9fdbc None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81612) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 02 09:12:57.978458 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG passlib.handlers.bcrypt [None req-0450b2c2-b8e8-4b22-9be7-ed5bafd9fdbc None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81612) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 02 09:12:58.038455 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0450b2c2-b8e8-4b22-9be7-ed5bafd9fdbc None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:12:58.040094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0450b2c2-b8e8-4b22-9be7-ed5bafd9fdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:12:58.172567 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 2/3] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:12:57 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 494 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 02 09:12:58.644468 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854605a7-cec2-41e3-890d-f8fcdd3897ba None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:12:58.712702 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: INFO dbcounter [None req-854605a7-cec2-41e3-890d-f8fcdd3897ba None None] Registered counter for database keystone Sep 02 09:12:58.731802 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG oslo_db.sqlalchemy.engines [None req-854605a7-cec2-41e3-890d-f8fcdd3897ba None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81611) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 02 09:12:58.737839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writer thread running {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 02 09:12:58.856616 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854605a7-cec2-41e3-890d-f8fcdd3897ba None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:12:58.860832 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d3093-ada5-448f-9a1f-520ab5d498e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:12:58.861451 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d3093-ada5-448f-9a1f-520ab5d498e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:58.861728 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d3093-ada5-448f-9a1f-520ab5d498e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:58.861993 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d3093-ada5-448f-9a1f-520ab5d498e8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:58.869898 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d3093-ada5-448f-9a1f-520ab5d498e8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:12:58.920855 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8a3d3093-ada5-448f-9a1f-520ab5d498e8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 02 09:12:58.921139 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d3093-ada5-448f-9a1f-520ab5d498e8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:12:58.930557 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 2/4] 10.222.0.57 () {58 vars in 1324 bytes} [Fri Sep 2 09:12:58 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 09:12:58.936546 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450b2c2-b8e8-4b22-9be7-ed5bafd9fdbc None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:12:58.951329 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0450b2c2-b8e8-4b22-9be7-ed5bafd9fdbc None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:12:58.956265 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb27b1-959b-4547-8ee0-24989cdab52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:12:58.957383 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb27b1-959b-4547-8ee0-24989cdab52e admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:12:58.957666 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb27b1-959b-4547-8ee0-24989cdab52e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:12:58.957940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb27b1-959b-4547-8ee0-24989cdab52e admin admin] PATH_INFO: `/v3/projects/ae5820815add48cd913951058cc9251c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:12:58.959540 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb27b1-959b-4547-8ee0-24989cdab52e admin admin] RBAC: Authorizing `identity:get_project(project_id=ae5820815add48cd913951058cc9251c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:12:59.011163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: WARNING keystone.common.rbac_enforcer.enforcer [None req-baeb27b1-959b-4547-8ee0-24989cdab52e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 02 09:12:59.011548 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb27b1-959b-4547-8ee0-24989cdab52e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:12:59.016647 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 3/5] 10.222.0.57 () {56 vars in 1207 bytes} [Fri Sep 2 09:12:58 2022] GET /identity/v3/projects/ae5820815add48cd913951058cc9251c => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:00.874260 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e90ed-9153-4a78-819a-39f7b914f8a1 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:00.874721 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e90ed-9153-4a78-819a-39f7b914f8a1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:00.874870 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48e90ed-9153-4a78-819a-39f7b914f8a1 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:00.879028 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 3/6] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:13:00.885154 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0ee30-7876-46c5-b142-480379b64caf None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:00.885623 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0ee30-7876-46c5-b142-480379b64caf None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:00.885946 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0ee30-7876-46c5-b142-480379b64caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:00.909632 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-76d0ee30-7876-46c5-b142-480379b64caf None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:00.911289 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-76d0ee30-7876-46c5-b142-480379b64caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:00.923880 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 4/7] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:00 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 02 09:13:01.188744 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e90ed-9153-4a78-819a-39f7b914f8a1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:01.201223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48e90ed-9153-4a78-819a-39f7b914f8a1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:01.205635 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117fc268-6014-46aa-a7e0-7b34377d82ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:01.206236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117fc268-6014-46aa-a7e0-7b34377d82ac admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:01.206516 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117fc268-6014-46aa-a7e0-7b34377d82ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:01.206821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117fc268-6014-46aa-a7e0-7b34377d82ac admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:01.215279 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117fc268-6014-46aa-a7e0-7b34377d82ac admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:01.217059 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117fc268-6014-46aa-a7e0-7b34377d82ac admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:01.225664 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 4/8] 10.222.0.57 () {58 vars in 1324 bytes} [Fri Sep 2 09:13:01 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 09:13:01.230949 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0ee30-7876-46c5-b142-480379b64caf None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:01.239896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0ee30-7876-46c5-b142-480379b64caf None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:01.244238 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5a22d8-6c14-4c56-8407-d277bb99171b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:01.244866 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a22d8-6c14-4c56-8407-d277bb99171b admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:01.245145 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a22d8-6c14-4c56-8407-d277bb99171b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:01.245416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a22d8-6c14-4c56-8407-d277bb99171b admin admin] PATH_INFO: `/v3/users/9cffe229f2f843e394c2248b000ecb37` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:01.246751 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5a22d8-6c14-4c56-8407-d277bb99171b admin admin] RBAC: Authorizing `identity:get_user(user_id=9cffe229f2f843e394c2248b000ecb37)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:01.248550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5a22d8-6c14-4c56-8407-d277bb99171b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:01.250043 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 5/9] 10.222.0.57 () {56 vars in 1198 bytes} [Fri Sep 2 09:13:01 2022] GET /identity/v3/users/9cffe229f2f843e394c2248b000ecb37 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:03.204868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f7245-6807-4706-b91a-c7e2dccb3eeb None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:03.205424 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f7245-6807-4706-b91a-c7e2dccb3eeb None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:03.205424 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f7245-6807-4706-b91a-c7e2dccb3eeb None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:03.206776 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 5/10] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:03.211859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01c864-979f-41c8-a35e-c52f03cc3319 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:03.212274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01c864-979f-41c8-a35e-c52f03cc3319 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:03.212417 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01c864-979f-41c8-a35e-c52f03cc3319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:03.230711 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ac01c864-979f-41c8-a35e-c52f03cc3319 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:03.231904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ac01c864-979f-41c8-a35e-c52f03cc3319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:03.252117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98114934-86b4-40b5-ac13-06316d356547 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:03.252403 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98114934-86b4-40b5-ac13-06316d356547 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:03.253034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98114934-86b4-40b5-ac13-06316d356547 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:03.253976 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 6/11] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:13:03.258993 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180fbcdd-c668-4c66-b3cc-16432b74d343 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:03.259278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180fbcdd-c668-4c66-b3cc-16432b74d343 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:03.259549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180fbcdd-c668-4c66-b3cc-16432b74d343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:03.301095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG passlib.handlers.bcrypt [None req-180fbcdd-c668-4c66-b3cc-16432b74d343 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81611) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 02 09:13:03.301490 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG passlib.handlers.bcrypt [None req-180fbcdd-c668-4c66-b3cc-16432b74d343 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81611) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 02 09:13:03.310431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 6/12] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 09:13:03.351108 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-180fbcdd-c668-4c66-b3cc-16432b74d343 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:03.352704 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-180fbcdd-c668-4c66-b3cc-16432b74d343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:03.450396 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 7/13] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 09:13:03.579301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01c864-979f-41c8-a35e-c52f03cc3319 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:03.602570 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:03.680698 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01c864-979f-41c8-a35e-c52f03cc3319 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:03.713888 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c5cdd-bb1c-4fe8-b77b-c484c7d0ddce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:03.714680 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c5cdd-bb1c-4fe8-b77b-c484c7d0ddce None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:03.714981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c5cdd-bb1c-4fe8-b77b-c484c7d0ddce None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:03.715253 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c5cdd-bb1c-4fe8-b77b-c484c7d0ddce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:03.715655 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180fbcdd-c668-4c66-b3cc-16432b74d343 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:03.726998 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c5cdd-bb1c-4fe8-b77b-c484c7d0ddce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:03.735560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-420c5cdd-bb1c-4fe8-b77b-c484c7d0ddce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:13:03.735560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:03.735560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:03.735560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:03.735560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:03.735560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:03.735560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:03.735560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:03.735560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:03.735560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:03.735560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:03.735560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:03.735560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:03.735560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:03.735560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:03.735560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:03.736498 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:03.736498 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:03.736498 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:03.736498 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:03.736498 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:03.736498 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:03.736498 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:03.736498 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:03.736498 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:03.736498 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:03.736498 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:03.736498 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:03.736498 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:03.736498 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:03.736498 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:03.736498 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:03.737416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:03.737416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:03.737416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:03.737416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:03.737416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:03.737416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:03.737416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:03.737416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:03.737416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:03.737416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:03.737416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:13:03.737416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:13:03.737416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 7/14] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:13:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:03.742408 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c5cdd-bb1c-4fe8-b77b-c484c7d0ddce None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:03.816290 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180fbcdd-c668-4c66-b3cc-16432b74d343 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:03.818539 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:03.823913 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:03.847456 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c5cdd-bb1c-4fe8-b77b-c484c7d0ddce None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:03.855446 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a19ea-8d4a-4b9a-9c8e-54d0ee128e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:03.856338 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a19ea-8d4a-4b9a-9c8e-54d0ee128e3e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:03.856621 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a19ea-8d4a-4b9a-9c8e-54d0ee128e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:03.856894 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a19ea-8d4a-4b9a-9c8e-54d0ee128e3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:03.880169 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc519dfc-5266-4be8-89c9-288baecc4b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:03.880786 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc519dfc-5266-4be8-89c9-288baecc4b40 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:03.881071 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc519dfc-5266-4be8-89c9-288baecc4b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:03.881349 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc519dfc-5266-4be8-89c9-288baecc4b40 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:03.882023 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc519dfc-5266-4be8-89c9-288baecc4b40 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:03.882330 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc519dfc-5266-4be8-89c9-288baecc4b40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:03.883750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc519dfc-5266-4be8-89c9-288baecc4b40 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:03.896214 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 8/15] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:03.901698 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc519dfc-5266-4be8-89c9-288baecc4b40 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:03.925636 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a19ea-8d4a-4b9a-9c8e-54d0ee128e3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:03.927300 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a19ea-8d4a-4b9a-9c8e-54d0ee128e3e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:04.002314 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc519dfc-5266-4be8-89c9-288baecc4b40 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:04.027351 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 8/16] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:13:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:13:04.033054 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a19ea-8d4a-4b9a-9c8e-54d0ee128e3e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:04.035316 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5b805-3e83-420b-8f04-e04773c3ab97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:04.035778 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:04.036791 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5b805-3e83-420b-8f04-e04773c3ab97 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:04.037078 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5b805-3e83-420b-8f04-e04773c3ab97 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:04.037357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf5b805-3e83-420b-8f04-e04773c3ab97 None admin] PATH_INFO: `/v3/users/9cffe229f2f843e394c2248b000ecb37` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:04.038531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf5b805-3e83-420b-8f04-e04773c3ab97 None admin] RBAC: Authorizing `identity:get_user(user_id=9cffe229f2f843e394c2248b000ecb37)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:04.040201 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf5b805-3e83-420b-8f04-e04773c3ab97 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:04.041791 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 9/17] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:03 2022] GET /identity/v3/users/9cffe229f2f843e394c2248b000ecb37 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:13:04.047647 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5b805-3e83-420b-8f04-e04773c3ab97 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:04.062579 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:04.135395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a19ea-8d4a-4b9a-9c8e-54d0ee128e3e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:04.146147 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf5b805-3e83-420b-8f04-e04773c3ab97 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:04.169438 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e517b-5e4d-4079-b861-7a40dffd9d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:04.170291 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e517b-5e4d-4079-b861-7a40dffd9d7e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:04.170571 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e517b-5e4d-4079-b861-7a40dffd9d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:04.170878 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e517b-5e4d-4079-b861-7a40dffd9d7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:04.180390 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c856c8-60bc-4f0e-b0e9-2a32c9f93772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:04.181011 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c856c8-60bc-4f0e-b0e9-2a32c9f93772 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:04.181289 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c856c8-60bc-4f0e-b0e9-2a32c9f93772 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:04.181559 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c856c8-60bc-4f0e-b0e9-2a32c9f93772 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:04.182666 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c856c8-60bc-4f0e-b0e9-2a32c9f93772 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:04.184673 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c856c8-60bc-4f0e-b0e9-2a32c9f93772 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:04.186198 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 10/18] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:13:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:13:04.189522 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e517b-5e4d-4079-b861-7a40dffd9d7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:04.191017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e517b-5e4d-4079-b861-7a40dffd9d7e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:04.191800 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c856c8-60bc-4f0e-b0e9-2a32c9f93772 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:04.209524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2f5e517b-5e4d-4079-b861-7a40dffd9d7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:13:04.209524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:04.209524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:04.209524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:04.209524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:04.209524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:04.209524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:04.209524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:04.209524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:04.209524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:04.209524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:04.209524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:04.209524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:13:04.209524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:13:04.209524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:13:04.209524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:13:04.210746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:04.210746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:04.210746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:04.210746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:04.210746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:04.210746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:04.210746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:04.210746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:04.210746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:04.210746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:04.210746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:04.210746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:04.210746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:04.210746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:04.210746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:04.210746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:04.210746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:13:04.211557 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:13:04.211557 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:13:04.211557 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:13:04.211557 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 9/19] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:13:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:13:04.216726 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e517b-5e4d-4079-b861-7a40dffd9d7e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:04.271104 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:04.274348 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:04.293075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c856c8-60bc-4f0e-b0e9-2a32c9f93772 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:04.315817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e517b-5e4d-4079-b861-7a40dffd9d7e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:04.326535 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f28d3-727c-4488-97d4-eadd50b8d70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:04.327171 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f28d3-727c-4488-97d4-eadd50b8d70b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:04.327451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f28d3-727c-4488-97d4-eadd50b8d70b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:04.327722 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f28d3-727c-4488-97d4-eadd50b8d70b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:04.328714 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f28d3-727c-4488-97d4-eadd50b8d70b None admin] RBAC: Adding query filter params (role.id=f886b8e8889549e896a8fb3426cce9e1, scope.domain.id=default, user.id=9cffe229f2f843e394c2248b000ecb37) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:04.329279 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f28d3-727c-4488-97d4-eadd50b8d70b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:04.330556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f28d3-727c-4488-97d4-eadd50b8d70b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:04.337098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 11/20] 10.222.0.57 () {56 vars in 1322 bytes} [Fri Sep 2 09:13:04 2022] GET /identity/v3/role_assignments?user.id=9cffe229f2f843e394c2248b000ecb37&scope.domain.id=default&role.id=f886b8e8889549e896a8fb3426cce9e1 => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:13:04.350836 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d21124c-698f-436d-8612-6522687c457c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:04.351474 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21124c-698f-436d-8612-6522687c457c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:04.351753 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21124c-698f-436d-8612-6522687c457c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:04.352021 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21124c-698f-436d-8612-6522687c457c None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:04.352686 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d21124c-698f-436d-8612-6522687c457c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:04.352990 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d21124c-698f-436d-8612-6522687c457c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:04.354411 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d21124c-698f-436d-8612-6522687c457c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:04.374600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 10/21] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:13:06.401197 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232ddf0-48ea-4f51-89d5-e125f59c0f18 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:06.401752 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232ddf0-48ea-4f51-89d5-e125f59c0f18 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:06.401752 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232ddf0-48ea-4f51-89d5-e125f59c0f18 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:06.403677 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 12/22] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:06.409490 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb1544-0bde-4cce-af56-ffd36b81e592 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:06.409917 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb1544-0bde-4cce-af56-ffd36b81e592 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:06.410092 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb1544-0bde-4cce-af56-ffd36b81e592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:06.435714 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3ebb1544-0bde-4cce-af56-ffd36b81e592 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:06.436927 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3ebb1544-0bde-4cce-af56-ffd36b81e592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:06.525339 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 11/23] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:06.537900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f0e5d-3f9e-49e1-960c-00e1bada6364 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:06.538184 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f0e5d-3f9e-49e1-960c-00e1bada6364 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:06.538464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f0e5d-3f9e-49e1-960c-00e1bada6364 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:06.539843 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 13/24] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:06.545142 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c49539-059f-4839-8fee-f258ac35cc1d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:06.545440 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c49539-059f-4839-8fee-f258ac35cc1d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:06.545717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c49539-059f-4839-8fee-f258ac35cc1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:06.575754 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-61c49539-059f-4839-8fee-f258ac35cc1d None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:06.576960 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-61c49539-059f-4839-8fee-f258ac35cc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:06.634818 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:06.661846 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 12/25] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:07.083070 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f0e5d-3f9e-49e1-960c-00e1bada6364 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:07.167270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ee50d-6da5-4ec3-80f3-ae4ab20db571 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:07.167635 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ee50d-6da5-4ec3-80f3-ae4ab20db571 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:07.167865 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ee50d-6da5-4ec3-80f3-ae4ab20db571 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:07.169223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 13/26] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:07.174941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48007e3d-a28a-4386-8b50-5efd9994b8ef None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:07.175229 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48007e3d-a28a-4386-8b50-5efd9994b8ef None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:07.175506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48007e3d-a28a-4386-8b50-5efd9994b8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:07.204550 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-48007e3d-a28a-4386-8b50-5efd9994b8ef None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:07.205537 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-48007e3d-a28a-4386-8b50-5efd9994b8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:07.232725 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f0e5d-3f9e-49e1-960c-00e1bada6364 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:07.268582 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f4147-f5cc-430d-b85e-b6ed4ebd0d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:07.269217 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f4147-f5cc-430d-b85e-b6ed4ebd0d8a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:07.269497 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f4147-f5cc-430d-b85e-b6ed4ebd0d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:07.269772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f4147-f5cc-430d-b85e-b6ed4ebd0d8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:07.281295 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f4147-f5cc-430d-b85e-b6ed4ebd0d8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:07.284048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 14/27] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 09:13:07.287310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-565f4147-f5cc-430d-b85e-b6ed4ebd0d8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:13:07.287310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:07.287310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:07.287310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:07.287310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:07.287310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:07.287310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:07.287310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:07.287310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:07.287310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:07.287310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:07.287310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:07.287310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:07.287310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:07.287310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:07.287310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:07.288157 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:07.288157 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:07.288157 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:07.288157 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:07.288157 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:07.288157 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:07.288157 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:07.288157 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:07.288157 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:07.288157 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:07.288157 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:07.288157 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:07.288157 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:07.288157 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:07.288157 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:07.288157 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:07.288916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:07.288916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:07.288916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:07.288916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:07.288916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:07.288916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:07.288916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:07.288916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:07.288916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:07.288916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:07.288916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:13:07.288916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:13:07.290760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 14/28] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:13:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:07.291674 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f4147-f5cc-430d-b85e-b6ed4ebd0d8a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:07.292446 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecc36c-6b6a-46c4-a6be-990f9d4f23e8 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:07.292740 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecc36c-6b6a-46c4-a6be-990f9d4f23e8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:07.293030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecc36c-6b6a-46c4-a6be-990f9d4f23e8 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:07.294430 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 15/29] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:07 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:07.301106 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04924618-fca2-42aa-97e1-ed212a936978 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:07.301386 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04924618-fca2-42aa-97e1-ed212a936978 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:07.301950 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04924618-fca2-42aa-97e1-ed212a936978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:07.306744 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:07.322651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-04924618-fca2-42aa-97e1-ed212a936978 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:07.325593 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-04924618-fca2-42aa-97e1-ed212a936978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:07.405310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 16/30] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 09:13:07.406165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04924618-fca2-42aa-97e1-ed212a936978 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:07.445906 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:07.488056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f4147-f5cc-430d-b85e-b6ed4ebd0d8a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:07.552333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0fde1-af2b-48c2-bd81-3a4d810da727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:07.553331 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0fde1-af2b-48c2-bd81-3a4d810da727 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:07.553833 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0fde1-af2b-48c2-bd81-3a4d810da727 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:07.554296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd0fde1-af2b-48c2-bd81-3a4d810da727 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:07.568404 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04924618-fca2-42aa-97e1-ed212a936978 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:07.572545 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd0fde1-af2b-48c2-bd81-3a4d810da727 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:07.579775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd0fde1-af2b-48c2-bd81-3a4d810da727 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:07.605772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-7cd0fde1-af2b-48c2-bd81-3a4d810da727 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:13:07.605772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:07.605772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:07.605772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:07.605772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:07.605772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:07.605772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:07.605772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:07.605772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:07.605772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:07.605772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:07.605772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:07.605772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:13:07.605772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:13:07.605772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:13:07.605772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:13:07.606671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:07.606671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:07.606671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:07.606671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:07.606671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:07.606671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:07.606671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:07.606671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:07.606671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:07.606671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:07.606671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:07.606671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:07.606671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:07.606671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:07.606671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:07.606671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:07.606671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:13:07.607490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:13:07.607490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:13:07.607490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:13:07.618036 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 15/31] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:13:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:07.625142 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0fde1-af2b-48c2-bd81-3a4d810da727 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:07.653184 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa76ed-9d97-4e6b-b744-245253ae63ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:07.654168 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa76ed-9d97-4e6b-b744-245253ae63ef None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:07.654726 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa76ed-9d97-4e6b-b744-245253ae63ef None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:07.655200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa76ed-9d97-4e6b-b744-245253ae63ef None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:07.656203 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa76ed-9d97-4e6b-b744-245253ae63ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:07.656705 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa76ed-9d97-4e6b-b744-245253ae63ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:07.659067 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa76ed-9d97-4e6b-b744-245253ae63ef None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:07.694269 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 17/32] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:07.696743 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa76ed-9d97-4e6b-b744-245253ae63ef None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:07.747001 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:07.813869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:07.818335 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa76ed-9d97-4e6b-b744-245253ae63ef None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:07.820525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd0fde1-af2b-48c2-bd81-3a4d810da727 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:07.857510 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d6b47-3931-4613-8afc-763b5ee02bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:07.858487 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d6b47-3931-4613-8afc-763b5ee02bc0 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:07.859064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d6b47-3931-4613-8afc-763b5ee02bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:07.859673 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d6b47-3931-4613-8afc-763b5ee02bc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:07.860554 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d6b47-3931-4613-8afc-763b5ee02bc0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:07.862188 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d6b47-3931-4613-8afc-763b5ee02bc0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:07.863451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6065206-91c9-4d27-adbe-fd688cb51bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:07.864131 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6065206-91c9-4d27-adbe-fd688cb51bad None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:07.864475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6065206-91c9-4d27-adbe-fd688cb51bad None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:07.864811 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6065206-91c9-4d27-adbe-fd688cb51bad None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:07.865527 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6065206-91c9-4d27-adbe-fd688cb51bad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:07.865898 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6065206-91c9-4d27-adbe-fd688cb51bad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:07.867735 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6065206-91c9-4d27-adbe-fd688cb51bad None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:07.888869 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 16/33] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:13:07.889664 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6065206-91c9-4d27-adbe-fd688cb51bad None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:08.000105 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6065206-91c9-4d27-adbe-fd688cb51bad None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:08.018012 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:08.036785 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551f6ec-3ac7-4ec6-b3ed-8b1a52c1c063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:08.037419 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551f6ec-3ac7-4ec6-b3ed-8b1a52c1c063 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:08.037701 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551f6ec-3ac7-4ec6-b3ed-8b1a52c1c063 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:08.037977 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551f6ec-3ac7-4ec6-b3ed-8b1a52c1c063 None admin] PATH_INFO: `/v3/users/9cffe229f2f843e394c2248b000ecb37` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:08.039195 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5551f6ec-3ac7-4ec6-b3ed-8b1a52c1c063 None admin] RBAC: Authorizing `identity:get_user(user_id=9cffe229f2f843e394c2248b000ecb37)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:08.040716 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5551f6ec-3ac7-4ec6-b3ed-8b1a52c1c063 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:08.046941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 17/34] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:07 2022] GET /identity/v3/users/9cffe229f2f843e394c2248b000ecb37 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:08.046941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551f6ec-3ac7-4ec6-b3ed-8b1a52c1c063 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:08.101533 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 18/35] 10.222.0.57 () {60 vars in 1131 bytes} [Fri Sep 2 09:13:07 2022] POST /identity/v3/roles => generated 219 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:13:08.102350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d6b47-3931-4613-8afc-763b5ee02bc0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:08.157950 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551f6ec-3ac7-4ec6-b3ed-8b1a52c1c063 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:08.197271 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb837f73-6cfe-492c-9869-23c3f8bb3dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:08.197897 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb837f73-6cfe-492c-9869-23c3f8bb3dc1 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:08.198185 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb837f73-6cfe-492c-9869-23c3f8bb3dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:08.198461 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb837f73-6cfe-492c-9869-23c3f8bb3dc1 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:08.199757 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb837f73-6cfe-492c-9869-23c3f8bb3dc1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:08.201173 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb837f73-6cfe-492c-9869-23c3f8bb3dc1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:08.244061 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d6b47-3931-4613-8afc-763b5ee02bc0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:08.296705 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a39c2-855c-4e32-bb4b-4aa31cbec3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:08.297242 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a39c2-855c-4e32-bb4b-4aa31cbec3b2 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:08.297533 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a39c2-855c-4e32-bb4b-4aa31cbec3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:08.297811 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a39c2-855c-4e32-bb4b-4aa31cbec3b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:08.298791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088a39c2-855c-4e32-bb4b-4aa31cbec3b2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:08.300382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088a39c2-855c-4e32-bb4b-4aa31cbec3b2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:08.369357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 18/36] 10.222.0.57 () {60 vars in 1131 bytes} [Fri Sep 2 09:13:08 2022] POST /identity/v3/roles => generated 225 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:13:08.371495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a878f-8dd6-4475-b7b0-da0d0f865383 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:08.371789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a878f-8dd6-4475-b7b0-da0d0f865383 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:08.372066 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a878f-8dd6-4475-b7b0-da0d0f865383 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:08.373515 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 19/37] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:08.374140 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a878f-8dd6-4475-b7b0-da0d0f865383 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:08.402539 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 19/38] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Sep 2 09:13:08 2022] POST /identity/v3/projects => generated 301 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:13:08.402539 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2f950-4af7-4925-ae9a-34dccbedd381 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:08.402539 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2f950-4af7-4925-ae9a-34dccbedd381 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:08.402539 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d2f950-4af7-4925-ae9a-34dccbedd381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:08.422641 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d2d2f950-4af7-4925-ae9a-34dccbedd381 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:08.424366 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d2d2f950-4af7-4925-ae9a-34dccbedd381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:08.439115 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:08.444057 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:08.519271 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a878f-8dd6-4475-b7b0-da0d0f865383 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:08.529964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 20/39] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:08.532136 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e997f-42de-414c-bd37-33e309825dd2 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:08.532441 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e997f-42de-414c-bd37-33e309825dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:08.532724 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e997f-42de-414c-bd37-33e309825dd2 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:08.534331 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 21/40] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:08.540317 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8c0ed-2aab-4ba2-b20e-64e94f515ecc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:08.540703 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8c0ed-2aab-4ba2-b20e-64e94f515ecc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:08.541057 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8c0ed-2aab-4ba2-b20e-64e94f515ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:08.562889 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1db8c0ed-2aab-4ba2-b20e-64e94f515ecc None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:08.564148 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1db8c0ed-2aab-4ba2-b20e-64e94f515ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:08.566922 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e92f3d-161c-443a-8b78-48b8a71174fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:08.568949 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e92f3d-161c-443a-8b78-48b8a71174fe None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:08.569427 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e92f3d-161c-443a-8b78-48b8a71174fe None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:08.569881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e92f3d-161c-443a-8b78-48b8a71174fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:08.571533 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e92f3d-161c-443a-8b78-48b8a71174fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:08.574040 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e92f3d-161c-443a-8b78-48b8a71174fe None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:08.576912 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 20/41] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:13:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:08.583238 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e92f3d-161c-443a-8b78-48b8a71174fe None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:08.654742 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 22/42] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:08.701182 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e92f3d-161c-443a-8b78-48b8a71174fe None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:08.715703 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:08.739310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3903df-d38b-4e04-aa8f-205cc01e36e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:08.739984 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3903df-d38b-4e04-aa8f-205cc01e36e1 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:08.740272 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3903df-d38b-4e04-aa8f-205cc01e36e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:08.740553 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3903df-d38b-4e04-aa8f-205cc01e36e1 None admin] PATH_INFO: `/v3/domains/default/users/9cffe229f2f843e394c2248b000ecb37/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:08.742772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3903df-d38b-4e04-aa8f-205cc01e36e1 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=9cffe229f2f843e394c2248b000ecb37, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:08.744262 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3903df-d38b-4e04-aa8f-205cc01e36e1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:08.818851 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8c0ed-2aab-4ba2-b20e-64e94f515ecc None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:08.857464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 21/43] 10.222.0.57 () {58 vars in 1361 bytes} [Fri Sep 2 09:13:08 2022] PUT /identity/v3/domains/default/users/9cffe229f2f843e394c2248b000ecb37/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:13:08.896913 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:08.931095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8c0ed-2aab-4ba2-b20e-64e94f515ecc None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:08.937879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3903df-d38b-4e04-aa8f-205cc01e36e1 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:08.978033 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba7ffc-c806-463e-886a-7edc9dbf9d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:08.978688 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba7ffc-c806-463e-886a-7edc9dbf9d0b None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:08.979006 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba7ffc-c806-463e-886a-7edc9dbf9d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:08.979288 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba7ffc-c806-463e-886a-7edc9dbf9d0b None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:08.980634 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba7ffc-c806-463e-886a-7edc9dbf9d0b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:08.982163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ba7ffc-c806-463e-886a-7edc9dbf9d0b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:09.053357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 23/44] 10.222.0.57 () {60 vars in 1131 bytes} [Fri Sep 2 09:13:08 2022] POST /identity/v3/roles => generated 223 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:13:09.160397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3903df-d38b-4e04-aa8f-205cc01e36e1 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:09.197292 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e29e0-4cca-4986-9ad6-d76274e18b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:09.197944 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e29e0-4cca-4986-9ad6-d76274e18b9c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:09.198235 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e29e0-4cca-4986-9ad6-d76274e18b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:09.199073 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e29e0-4cca-4986-9ad6-d76274e18b9c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:09.200220 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e29e0-4cca-4986-9ad6-d76274e18b9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:09.201678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e29e0-4cca-4986-9ad6-d76274e18b9c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:09.203556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 22/45] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:13:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:13:09.209139 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba7ffc-c806-463e-886a-7edc9dbf9d0b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:09.311919 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba7ffc-c806-463e-886a-7edc9dbf9d0b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:09.340306 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:09.345892 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98681a-d245-4850-bcb7-57d74175bf75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:09.346507 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98681a-d245-4850-bcb7-57d74175bf75 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:09.346815 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98681a-d245-4850-bcb7-57d74175bf75 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:09.347088 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98681a-d245-4850-bcb7-57d74175bf75 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:09.347744 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f98681a-d245-4850-bcb7-57d74175bf75 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:09.349187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f98681a-d245-4850-bcb7-57d74175bf75 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:09.470258 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 24/46] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Sep 2 09:13:09 2022] POST /identity/v3/projects => generated 312 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:13:10.774570 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e4654-d394-4c2f-8c49-9b3ec2238da3 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:10.775173 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e4654-d394-4c2f-8c49-9b3ec2238da3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:10.775173 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e4654-d394-4c2f-8c49-9b3ec2238da3 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:10.777114 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 23/47] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:10.782380 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a30f39-0fa7-4f1d-b1a5-ba0dd235f504 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:10.782829 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a30f39-0fa7-4f1d-b1a5-ba0dd235f504 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:10.782971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a30f39-0fa7-4f1d-b1a5-ba0dd235f504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:10.800525 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-48a30f39-0fa7-4f1d-b1a5-ba0dd235f504 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:10.801696 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-48a30f39-0fa7-4f1d-b1a5-ba0dd235f504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:10.866486 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 25/48] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:11.134537 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e4654-d394-4c2f-8c49-9b3ec2238da3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:11.175599 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:11.232432 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e4654-d394-4c2f-8c49-9b3ec2238da3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:11.266464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9a24f1-ac81-40ea-9c23-abd1435cae89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:11.267136 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a24f1-ac81-40ea-9c23-abd1435cae89 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:11.267450 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a24f1-ac81-40ea-9c23-abd1435cae89 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:11.267727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9a24f1-ac81-40ea-9c23-abd1435cae89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:11.277788 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9a24f1-ac81-40ea-9c23-abd1435cae89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:11.282693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-5b9a24f1-ac81-40ea-9c23-abd1435cae89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:13:11.282693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:11.282693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:11.282693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:11.282693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:11.282693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:11.282693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:11.282693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:11.282693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:11.282693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:11.282693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:11.282693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:11.282693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:11.282693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:11.282693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:11.282693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:11.283525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:11.283525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:11.283525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:11.283525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:11.283525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:11.283525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:11.283525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:11.283525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:11.283525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:11.283525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:11.283525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:11.283525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:11.283525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:11.283525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:11.283525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:11.283525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:11.284517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:11.284517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:11.284517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:11.284517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:11.284517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:11.284517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:11.284517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:11.284517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:11.284517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:11.284517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:11.284517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:13:11.284517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:13:11.284517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 24/49] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:13:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:11.289678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a30f39-0fa7-4f1d-b1a5-ba0dd235f504 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:11.360661 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f039a-9f44-4fe5-bcba-5b5fec40ad12 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:11.360953 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f039a-9f44-4fe5-bcba-5b5fec40ad12 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:11.361237 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f039a-9f44-4fe5-bcba-5b5fec40ad12 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:11.362605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 25/50] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:13:11.367351 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ea27e-832c-404a-9551-795dab86e289 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:11.367637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ea27e-832c-404a-9551-795dab86e289 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:11.367912 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ea27e-832c-404a-9551-795dab86e289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:11.387268 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e72ea27e-832c-404a-9551-795dab86e289 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:11.388734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e72ea27e-832c-404a-9551-795dab86e289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:11.392559 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a30f39-0fa7-4f1d-b1a5-ba0dd235f504 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:11.406109 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:11.426456 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858e803-4d15-4741-9a39-0c34e2d4a98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:11.427105 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858e803-4d15-4741-9a39-0c34e2d4a98e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:11.427390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858e803-4d15-4741-9a39-0c34e2d4a98e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:11.427670 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858e803-4d15-4741-9a39-0c34e2d4a98e None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:11.428341 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858e803-4d15-4741-9a39-0c34e2d4a98e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:11.428649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858e803-4d15-4741-9a39-0c34e2d4a98e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:11.430036 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858e803-4d15-4741-9a39-0c34e2d4a98e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:11.438527 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 26/51] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:11.444158 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858e803-4d15-4741-9a39-0c34e2d4a98e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:11.460005 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 26/52] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 09:13:11.644939 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858e803-4d15-4741-9a39-0c34e2d4a98e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:11.678017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499fde83-7bbf-4236-ad57-5c1849938737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:11.678649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499fde83-7bbf-4236-ad57-5c1849938737 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:11.678970 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499fde83-7bbf-4236-ad57-5c1849938737 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:11.679247 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499fde83-7bbf-4236-ad57-5c1849938737 None admin] PATH_INFO: `/v3/users/9cffe229f2f843e394c2248b000ecb37` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:11.680690 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499fde83-7bbf-4236-ad57-5c1849938737 None admin] RBAC: Authorizing `identity:get_user(user_id=9cffe229f2f843e394c2248b000ecb37)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:11.682105 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499fde83-7bbf-4236-ad57-5c1849938737 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:11.684272 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 27/53] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:11 2022] GET /identity/v3/users/9cffe229f2f843e394c2248b000ecb37 => generated 247 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:13:11.690068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ea27e-832c-404a-9551-795dab86e289 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:11.719459 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:11.731841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499fde83-7bbf-4236-ad57-5c1849938737 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:11.775837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:11.789306 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ea27e-832c-404a-9551-795dab86e289 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:11.822253 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23cd21-9583-40e0-9954-f0cdb5c17745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:11.822903 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23cd21-9583-40e0-9954-f0cdb5c17745 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:11.823186 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23cd21-9583-40e0-9954-f0cdb5c17745 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:11.823527 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23cd21-9583-40e0-9954-f0cdb5c17745 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:11.824629 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d23cd21-9583-40e0-9954-f0cdb5c17745 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:11.826068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d23cd21-9583-40e0-9954-f0cdb5c17745 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:11.827634 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 27/54] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:13:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:11.833697 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23cd21-9583-40e0-9954-f0cdb5c17745 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:11.833953 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499fde83-7bbf-4236-ad57-5c1849938737 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:11.870402 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038cce2f-f75c-425d-87c8-994bd8d88bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:11.871265 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038cce2f-f75c-425d-87c8-994bd8d88bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:11.871687 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038cce2f-f75c-425d-87c8-994bd8d88bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:11.872100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038cce2f-f75c-425d-87c8-994bd8d88bc8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:11.873398 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038cce2f-f75c-425d-87c8-994bd8d88bc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:11.875062 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038cce2f-f75c-425d-87c8-994bd8d88bc8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:11.876775 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 28/55] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:13:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:13:11.882193 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038cce2f-f75c-425d-87c8-994bd8d88bc8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:11.930971 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:11.935150 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23cd21-9583-40e0-9954-f0cdb5c17745 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:11.969513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a8db54-e411-420b-bad0-10826bfa8be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:11.970137 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8db54-e411-420b-bad0-10826bfa8be0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:11.970421 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8db54-e411-420b-bad0-10826bfa8be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:11.970713 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a8db54-e411-420b-bad0-10826bfa8be0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:11.971457 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a8db54-e411-420b-bad0-10826bfa8be0 None admin] RBAC: Adding query filter params (role.id=f886b8e8889549e896a8fb3426cce9e1, scope.domain.id=default, user.id=9cffe229f2f843e394c2248b000ecb37) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:11.971799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a8db54-e411-420b-bad0-10826bfa8be0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:11.973255 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a8db54-e411-420b-bad0-10826bfa8be0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:11.978911 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 28/56] 10.222.0.57 () {56 vars in 1322 bytes} [Fri Sep 2 09:13:11 2022] GET /identity/v3/role_assignments?user.id=9cffe229f2f843e394c2248b000ecb37&scope.domain.id=default&role.id=f886b8e8889549e896a8fb3426cce9e1 => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:13:11.981090 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038cce2f-f75c-425d-87c8-994bd8d88bc8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:11.992355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:12.018134 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9cb9c3-a6a0-41e4-95c7-acf757328256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:12.018798 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cb9c3-a6a0-41e4-95c7-acf757328256 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:12.019087 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cb9c3-a6a0-41e4-95c7-acf757328256 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:12.019362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9cb9c3-a6a0-41e4-95c7-acf757328256 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:12.020060 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9cb9c3-a6a0-41e4-95c7-acf757328256 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:12.021544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9cb9c3-a6a0-41e4-95c7-acf757328256 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:12.111597 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 29/57] 10.222.0.57 () {60 vars in 1140 bytes} [Fri Sep 2 09:13:11 2022] POST /identity/v3/projects => generated 298 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:13:14.001746 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcfdc2-cc56-4954-936c-44423f9a9c99 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:14.002283 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcfdc2-cc56-4954-936c-44423f9a9c99 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:14.002283 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dcfdc2-cc56-4954-936c-44423f9a9c99 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:14.003689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 29/58] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:14.008774 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9f2dd-7c3c-4573-aa0a-745daf0208ac None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:14.009179 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9f2dd-7c3c-4573-aa0a-745daf0208ac None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:14.009340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b9f2dd-7c3c-4573-aa0a-745daf0208ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:14.028106 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a7b9f2dd-7c3c-4573-aa0a-745daf0208ac None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:14.029314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a7b9f2dd-7c3c-4573-aa0a-745daf0208ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:14.099237 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 30/59] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:14.369179 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dcfdc2-cc56-4954-936c-44423f9a9c99 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:14.469128 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dcfdc2-cc56-4954-936c-44423f9a9c99 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:14.508116 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace2877d-5840-471f-a932-6dbef7c554be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:14.508738 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2877d-5840-471f-a932-6dbef7c554be None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:14.509026 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2877d-5840-471f-a932-6dbef7c554be None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:14.509306 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace2877d-5840-471f-a932-6dbef7c554be None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:14.510437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace2877d-5840-471f-a932-6dbef7c554be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:14.511948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace2877d-5840-471f-a932-6dbef7c554be None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:14.513515 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 30/60] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:13:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:14.518914 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9f2dd-7c3c-4573-aa0a-745daf0208ac None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:14.605633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:14.613848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b9f2dd-7c3c-4573-aa0a-745daf0208ac None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:14.646658 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eddb446-f0bb-4079-8f77-da790ac4a4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:14.646929 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eddb446-f0bb-4079-8f77-da790ac4a4e7 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:14.647213 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eddb446-f0bb-4079-8f77-da790ac4a4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:14.647488 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eddb446-f0bb-4079-8f77-da790ac4a4e7 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:14.648132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eddb446-f0bb-4079-8f77-da790ac4a4e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:14.649576 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eddb446-f0bb-4079-8f77-da790ac4a4e7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:14.752795 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 31/61] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Sep 2 09:13:14 2022] POST /identity/v3/users => generated 275 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:13:17.803668 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edbc7b-b459-416a-b08c-6a9206639a6d None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:17.804179 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edbc7b-b459-416a-b08c-6a9206639a6d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:17.804252 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edbc7b-b459-416a-b08c-6a9206639a6d None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:17.805587 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 31/62] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:17.811361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320250b-ee99-4f9a-8623-dd37ec3e929b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:17.811749 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320250b-ee99-4f9a-8623-dd37ec3e929b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:17.811917 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3320250b-ee99-4f9a-8623-dd37ec3e929b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:17.837546 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3320250b-ee99-4f9a-8623-dd37ec3e929b None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:17.838767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3320250b-ee99-4f9a-8623-dd37ec3e929b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:17.900762 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbba64c-8d2f-4e14-ac1a-2c66d44e7df6 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:17.901087 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbba64c-8d2f-4e14-ac1a-2c66d44e7df6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:17.901325 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbba64c-8d2f-4e14-ac1a-2c66d44e7df6 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:17.902936 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 32/63] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:13:17.908249 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ea256-2d96-48d9-8774-2fa7ddc03813 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:17.908535 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ea256-2d96-48d9-8774-2fa7ddc03813 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:17.909063 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ea256-2d96-48d9-8774-2fa7ddc03813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:17.936995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d69ea256-2d96-48d9-8774-2fa7ddc03813 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:17.938203 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d69ea256-2d96-48d9-8774-2fa7ddc03813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:17.945270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 32/64] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:17.971172 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:17.998007 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e3d55-e4fc-4e30-b4d1-fa26751b6c22 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:17.998325 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e3d55-e4fc-4e30-b4d1-fa26751b6c22 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:17.998581 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e3d55-e4fc-4e30-b4d1-fa26751b6c22 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:17.999968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 33/65] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:13:18.005100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e7dc9-ea2b-45ff-99b1-31dbbbb61ad7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:18.005393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e7dc9-ea2b-45ff-99b1-31dbbbb61ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:18.005672 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e7dc9-ea2b-45ff-99b1-31dbbbb61ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:18.034410 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-128e7dc9-ea2b-45ff-99b1-31dbbbb61ad7 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:18.036902 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-128e7dc9-ea2b-45ff-99b1-31dbbbb61ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:18.058198 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 33/66] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:18.123726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f0b1a-3705-421d-ae5c-ed15a7b1f20d None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:18.124057 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f0b1a-3705-421d-ae5c-ed15a7b1f20d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:18.124296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f0b1a-3705-421d-ae5c-ed15a7b1f20d None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:18.125653 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 34/67] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:13:18.130626 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554ab4d-cbd9-4f46-a69c-b629b21254b7 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:18.131161 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554ab4d-cbd9-4f46-a69c-b629b21254b7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:18.131656 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554ab4d-cbd9-4f46-a69c-b629b21254b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:18.149733 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 34/68] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 09:13:18.155560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f554ab4d-cbd9-4f46-a69c-b629b21254b7 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:18.156757 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f554ab4d-cbd9-4f46-a69c-b629b21254b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:18.248556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 35/69] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 09:13:18.317979 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e7dc9-ea2b-45ff-99b1-31dbbbb61ad7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:18.329451 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:18.437800 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554ab4d-cbd9-4f46-a69c-b629b21254b7 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:18.442737 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e7dc9-ea2b-45ff-99b1-31dbbbb61ad7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:18.480170 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef65f0e-eada-4b25-951d-31f8163be178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:18.480847 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef65f0e-eada-4b25-951d-31f8163be178 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:18.481137 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef65f0e-eada-4b25-951d-31f8163be178 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:18.481419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef65f0e-eada-4b25-951d-31f8163be178 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:18.494066 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef65f0e-eada-4b25-951d-31f8163be178 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:18.501617 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-3ef65f0e-eada-4b25-951d-31f8163be178 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:13:18.501617 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:18.501617 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:18.501617 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:18.501617 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:18.501617 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:18.501617 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:18.501617 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:18.501617 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:18.501617 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:18.501617 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:18.501617 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:18.501617 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:18.501617 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:18.501617 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:18.501617 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:18.517595 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:18.517595 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:18.517595 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:18.517595 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:18.517595 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:18.517595 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:18.517595 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:18.517595 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:18.517595 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:18.517595 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:18.517595 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:18.517595 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:18.517595 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:18.517595 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:18.517595 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:18.517595 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:18.518575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:18.518575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:18.518575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:18.518575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:18.518575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:18.518575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:18.518575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:18.518575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:18.518575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:18.518575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:18.518575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:13:18.518575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:13:18.518575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 35/70] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:13:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:18.518575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef65f0e-eada-4b25-951d-31f8163be178 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:18.542931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:18.554648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554ab4d-cbd9-4f46-a69c-b629b21254b7 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:18.584354 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:18.592225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ec0cc-7c55-4e03-b05a-37b7dee6e4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:18.593043 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ec0cc-7c55-4e03-b05a-37b7dee6e4f3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:18.593523 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ec0cc-7c55-4e03-b05a-37b7dee6e4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:18.593941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ec0cc-7c55-4e03-b05a-37b7dee6e4f3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:18.605195 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9ec0cc-7c55-4e03-b05a-37b7dee6e4f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:18.610965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-2f9ec0cc-7c55-4e03-b05a-37b7dee6e4f3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:18.610965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:18.610965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:18.610965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:18.610965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:18.610965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:18.610965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:18.610965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:18.610965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:18.610965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:18.610965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:18.610965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:18.610965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:18.610965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:18.610965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:18.610965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:18.611991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:18.611991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:18.611991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:18.611991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:18.611991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:18.611991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:18.611991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:18.611991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:18.611991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:18.611991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:18.611991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:18.611991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:18.611991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:18.611991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:18.611991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:18.611991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:18.612939 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:18.612939 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:18.612939 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:18.612939 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:18.612939 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:18.612939 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:18.612939 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:18.612939 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:18.612939 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:18.612939 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:18.612939 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:18.612939 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:13:18.614857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 36/71] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:13:18.615950 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ec0cc-7c55-4e03-b05a-37b7dee6e4f3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:18.625779 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef65f0e-eada-4b25-951d-31f8163be178 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:18.659544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b4e66-0e64-479d-a794-b84da902a595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:18.660177 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b4e66-0e64-479d-a794-b84da902a595 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:18.660461 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b4e66-0e64-479d-a794-b84da902a595 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:18.660732 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b4e66-0e64-479d-a794-b84da902a595 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:18.671596 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1b4e66-0e64-479d-a794-b84da902a595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:18.677953 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-9a1b4e66-0e64-479d-a794-b84da902a595 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 09:13:18.677953 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:18.677953 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:18.677953 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:18.677953 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:18.677953 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:18.677953 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:18.677953 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:18.677953 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:18.677953 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:18.677953 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:18.677953 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:18.677953 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:18.677953 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:18.677953 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:18.677953 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:18.678864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:18.678864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:18.678864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:18.678864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:18.678864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:18.678864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:18.678864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:18.678864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:18.678864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:18.678864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:18.678864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:18.678864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:18.678864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:18.678864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:18.678864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:18.678864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:18.679874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:18.679874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:18.679874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:18.679874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:18.679874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:18.679874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:18.679874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:18.679874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:18.679874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:18.679874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:18.679874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 09:13:18.679874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:13:18.679874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 36/72] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Sep 2 09:13:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:18.680751 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aef084-18d7-44f8-aa84-df9565cd0f4b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:18.680751 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aef084-18d7-44f8-aa84-df9565cd0f4b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:18.680863 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aef084-18d7-44f8-aa84-df9565cd0f4b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:18.682267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 37/73] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:18.682841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aef084-18d7-44f8-aa84-df9565cd0f4b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:18.728273 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ec0cc-7c55-4e03-b05a-37b7dee6e4f3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:18.762758 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d36ae4-b383-4cf2-b6cd-0f83d24d353e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:18.763363 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d36ae4-b383-4cf2-b6cd-0f83d24d353e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:18.763655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d36ae4-b383-4cf2-b6cd-0f83d24d353e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:18.763925 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d36ae4-b383-4cf2-b6cd-0f83d24d353e None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:18.764586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d36ae4-b383-4cf2-b6cd-0f83d24d353e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:18.764892 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d36ae4-b383-4cf2-b6cd-0f83d24d353e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:18.766277 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d36ae4-b383-4cf2-b6cd-0f83d24d353e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:18.774203 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 37/74] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:18.775015 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d36ae4-b383-4cf2-b6cd-0f83d24d353e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:18.785550 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aef084-18d7-44f8-aa84-df9565cd0f4b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:18.792773 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:18.812244 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:18.819860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f5b6c5-7944-42db-ad47-59ccbda392bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:18.820477 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5b6c5-7944-42db-ad47-59ccbda392bf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:18.820761 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5b6c5-7944-42db-ad47-59ccbda392bf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:18.821044 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5b6c5-7944-42db-ad47-59ccbda392bf None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:18.821751 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f5b6c5-7944-42db-ad47-59ccbda392bf None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:18.822059 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f5b6c5-7944-42db-ad47-59ccbda392bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:18.823478 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f5b6c5-7944-42db-ad47-59ccbda392bf None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:18.832667 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 38/75] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Sep 2 09:13:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:18.833493 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f5b6c5-7944-42db-ad47-59ccbda392bf None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:18.883186 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d36ae4-b383-4cf2-b6cd-0f83d24d353e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:18.917057 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a4940-7967-42a1-b8a2-6140928f8530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:18.917702 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a4940-7967-42a1-b8a2-6140928f8530 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:18.917989 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a4940-7967-42a1-b8a2-6140928f8530 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:18.918821 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a4940-7967-42a1-b8a2-6140928f8530 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:18.928762 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a4940-7967-42a1-b8a2-6140928f8530 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:18.931393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f5b6c5-7944-42db-ad47-59ccbda392bf None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:18.934576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-3b8a4940-7967-42a1-b8a2-6140928f8530 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:18.934576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:18.934576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:18.934576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:18.934576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:18.934576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:18.934576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:18.934576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:18.934576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:18.934576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:18.934576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:18.934576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:18.934576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:18.934576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:18.934576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:18.934576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:18.935466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:18.935466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:18.935466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:18.935466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:18.935466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:18.935466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:18.935466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:18.935466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:18.935466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:18.935466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:18.935466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:18.935466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:18.935466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:18.935466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:18.935466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:18.935466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:18.936223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:18.936223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:18.936223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:18.936223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:18.936223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:18.936223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:18.936223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:18.936223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:18.936223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:18.936223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:18.936223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:18.936223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:13:18.936223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 38/76] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:18.936975 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe088b-a089-4a75-947a-8f113b5d0b16 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:18.937313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe088b-a089-4a75-947a-8f113b5d0b16 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:18.937542 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe088b-a089-4a75-947a-8f113b5d0b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:18.954984 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8cbe088b-a089-4a75-947a-8f113b5d0b16 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:18.956185 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8cbe088b-a089-4a75-947a-8f113b5d0b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:18.969626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb0b53-e346-4e55-b2d7-331d7545ccec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:18.970427 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb0b53-e346-4e55-b2d7-331d7545ccec None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:18.970732 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb0b53-e346-4e55-b2d7-331d7545ccec None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:18.971028 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb0b53-e346-4e55-b2d7-331d7545ccec None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:18.971702 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edb0b53-e346-4e55-b2d7-331d7545ccec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:18.972093 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edb0b53-e346-4e55-b2d7-331d7545ccec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:18.973375 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edb0b53-e346-4e55-b2d7-331d7545ccec None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:18.981671 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 39/77] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:13:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:18.982492 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb0b53-e346-4e55-b2d7-331d7545ccec None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:19.024875 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 39/78] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:19.025708 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe088b-a089-4a75-947a-8f113b5d0b16 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:19.035863 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:19.039577 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:19.091866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb0b53-e346-4e55-b2d7-331d7545ccec None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:19.127679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c18d2-b6c9-4ff7-b9f7-2d4fb0280ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:19.127865 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe088b-a089-4a75-947a-8f113b5d0b16 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:19.128647 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c18d2-b6c9-4ff7-b9f7-2d4fb0280ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:19.129187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c18d2-b6c9-4ff7-b9f7-2d4fb0280ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:19.129635 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c18d2-b6c9-4ff7-b9f7-2d4fb0280ec0 None admin] PATH_INFO: `/v3/users/9cffe229f2f843e394c2248b000ecb37` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:19.131005 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c18d2-b6c9-4ff7-b9f7-2d4fb0280ec0 None admin] RBAC: Authorizing `identity:get_user(user_id=9cffe229f2f843e394c2248b000ecb37)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:19.132618 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c18d2-b6c9-4ff7-b9f7-2d4fb0280ec0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:19.134395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 40/79] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:18 2022] GET /identity/v3/users/9cffe229f2f843e394c2248b000ecb37 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:19.135339 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c18d2-b6c9-4ff7-b9f7-2d4fb0280ec0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:19.165729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0df3f0-650b-40fa-b317-921f0ba8bc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:19.166364 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0df3f0-650b-40fa-b317-921f0ba8bc1a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:19.166649 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0df3f0-650b-40fa-b317-921f0ba8bc1a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:19.166956 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0df3f0-650b-40fa-b317-921f0ba8bc1a None admin] PATH_INFO: `/v3/users/604723d5b48f4e869bccbe56651aa80b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:19.180440 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0df3f0-650b-40fa-b317-921f0ba8bc1a None admin] RBAC: Authorizing `identity:get_user(user_id=604723d5b48f4e869bccbe56651aa80b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:19.181894 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0df3f0-650b-40fa-b317-921f0ba8bc1a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:19.183547 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 40/80] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:19 2022] GET /identity/v3/users/604723d5b48f4e869bccbe56651aa80b => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:19.184999 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0df3f0-650b-40fa-b317-921f0ba8bc1a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:19.242903 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:19.247714 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c18d2-b6c9-4ff7-b9f7-2d4fb0280ec0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:19.268803 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:19.284348 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfbc48f-e513-457e-a54c-17c89bea5edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:19.284615 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0df3f0-650b-40fa-b317-921f0ba8bc1a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:19.285028 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfbc48f-e513-457e-a54c-17c89bea5edb None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:19.285309 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfbc48f-e513-457e-a54c-17c89bea5edb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:19.285581 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfbc48f-e513-457e-a54c-17c89bea5edb None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:19.286457 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfbc48f-e513-457e-a54c-17c89bea5edb None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:19.286918 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfbc48f-e513-457e-a54c-17c89bea5edb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:19.290372 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfbc48f-e513-457e-a54c-17c89bea5edb None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:19.297290 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 41/81] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Sep 2 09:13:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:19.298095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfbc48f-e513-457e-a54c-17c89bea5edb None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:19.323230 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea9ea0-31e1-4ca6-9f7a-1b802fdde513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:19.323874 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea9ea0-31e1-4ca6-9f7a-1b802fdde513 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:19.324156 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea9ea0-31e1-4ca6-9f7a-1b802fdde513 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:19.324980 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea9ea0-31e1-4ca6-9f7a-1b802fdde513 None admin] PATH_INFO: `/v3/users/604723d5b48f4e869bccbe56651aa80b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:19.326112 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdea9ea0-31e1-4ca6-9f7a-1b802fdde513 None admin] RBAC: Authorizing `identity:get_user(user_id=604723d5b48f4e869bccbe56651aa80b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:19.327825 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdea9ea0-31e1-4ca6-9f7a-1b802fdde513 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:19.329390 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 41/82] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:19 2022] GET /identity/v3/users/604723d5b48f4e869bccbe56651aa80b => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:19.330148 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea9ea0-31e1-4ca6-9f7a-1b802fdde513 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:19.401039 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfbc48f-e513-457e-a54c-17c89bea5edb None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:19.425986 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea9ea0-31e1-4ca6-9f7a-1b802fdde513 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:19.433596 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a08aa-7f04-4f61-8ad4-231669f5bbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:19.434697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a08aa-7f04-4f61-8ad4-231669f5bbe6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:19.435007 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a08aa-7f04-4f61-8ad4-231669f5bbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:19.435281 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a08aa-7f04-4f61-8ad4-231669f5bbe6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:19.452980 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:19.460671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6104d-8a30-44fa-9ad0-bea716f39654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:19.463095 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6104d-8a30-44fa-9ad0-bea716f39654 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:19.463095 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6104d-8a30-44fa-9ad0-bea716f39654 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:19.463095 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6104d-8a30-44fa-9ad0-bea716f39654 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:19.483537 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:19.501950 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a08aa-7f04-4f61-8ad4-231669f5bbe6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:19.503458 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899a08aa-7f04-4f61-8ad4-231669f5bbe6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:19.529043 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6104d-8a30-44fa-9ad0-bea716f39654 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:19.530525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6104d-8a30-44fa-9ad0-bea716f39654 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:19.600416 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 42/83] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:13:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:13:19.601249 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a08aa-7f04-4f61-8ad4-231669f5bbe6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:19.627692 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 42/84] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:13:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:13:19.628525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6104d-8a30-44fa-9ad0-bea716f39654 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:19.656373 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:19.680871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:19.699156 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a08aa-7f04-4f61-8ad4-231669f5bbe6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:19.733088 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6104d-8a30-44fa-9ad0-bea716f39654 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:19.733278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064ff1c-df61-4912-a9f7-31deba5e3240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:19.733936 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064ff1c-df61-4912-a9f7-31deba5e3240 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:19.734289 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064ff1c-df61-4912-a9f7-31deba5e3240 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:19.734642 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064ff1c-df61-4912-a9f7-31deba5e3240 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:19.735802 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b064ff1c-df61-4912-a9f7-31deba5e3240 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:19.737245 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b064ff1c-df61-4912-a9f7-31deba5e3240 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:19.738890 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 43/85] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:13:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:19.739665 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064ff1c-df61-4912-a9f7-31deba5e3240 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:19.767733 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a7f38-033c-49eb-a040-1a09b17584b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:19.768365 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a7f38-033c-49eb-a040-1a09b17584b7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:19.768649 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a7f38-033c-49eb-a040-1a09b17584b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:19.768927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a7f38-033c-49eb-a040-1a09b17584b7 None admin] PATH_INFO: `/v3/users/604723d5b48f4e869bccbe56651aa80b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:19.770048 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004a7f38-033c-49eb-a040-1a09b17584b7 None admin] RBAC: Authorizing `identity:get_user(user_id=604723d5b48f4e869bccbe56651aa80b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:19.771517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004a7f38-033c-49eb-a040-1a09b17584b7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:19.773070 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 43/86] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:19 2022] GET /identity/v3/users/604723d5b48f4e869bccbe56651aa80b => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:19.773824 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a7f38-033c-49eb-a040-1a09b17584b7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:19.838769 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064ff1c-df61-4912-a9f7-31deba5e3240 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:19.865721 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:19.872151 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a7f38-033c-49eb-a040-1a09b17584b7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:19.873041 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99945ea-4b1d-4ddf-951f-039b812d83b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:19.873658 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99945ea-4b1d-4ddf-951f-039b812d83b3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:19.873941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99945ea-4b1d-4ddf-951f-039b812d83b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:19.874214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99945ea-4b1d-4ddf-951f-039b812d83b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:19.892580 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:19.905672 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd81843-984d-4307-820c-2d730d03cd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:19.906306 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd81843-984d-4307-820c-2d730d03cd9a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:19.906636 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd81843-984d-4307-820c-2d730d03cd9a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:19.906902 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd81843-984d-4307-820c-2d730d03cd9a None admin] PATH_INFO: `/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:19.908117 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd81843-984d-4307-820c-2d730d03cd9a None admin] RBAC: Authorizing `identity:get_project(project_id=79a2aa8010c54ad7ad7b27f08f4de74c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:19.909604 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd81843-984d-4307-820c-2d730d03cd9a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:19.911262 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 44/87] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:19 2022] GET /identity/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:19.912032 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd81843-984d-4307-820c-2d730d03cd9a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:19.941404 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99945ea-4b1d-4ddf-951f-039b812d83b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:19.943071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99945ea-4b1d-4ddf-951f-039b812d83b3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:20.008853 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd81843-984d-4307-820c-2d730d03cd9a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:20.037789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 44/88] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:13:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:13:20.038680 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99945ea-4b1d-4ddf-951f-039b812d83b3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:20.046794 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ab16a-900d-4104-a96d-7813ec71d48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:20.047432 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ab16a-900d-4104-a96d-7813ec71d48a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:20.047726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ab16a-900d-4104-a96d-7813ec71d48a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:20.048007 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ab16a-900d-4104-a96d-7813ec71d48a None admin] PATH_INFO: `/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:20.049134 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ab16a-900d-4104-a96d-7813ec71d48a None admin] RBAC: Authorizing `identity:get_project(project_id=79a2aa8010c54ad7ad7b27f08f4de74c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:20.050569 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ab16a-900d-4104-a96d-7813ec71d48a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:20.052251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 45/89] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:19 2022] GET /identity/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:20.053008 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ab16a-900d-4104-a96d-7813ec71d48a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:20.082738 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:20.090804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:20.138207 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99945ea-4b1d-4ddf-951f-039b812d83b3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:20.150162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ab16a-900d-4104-a96d-7813ec71d48a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:20.174583 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a74137-85de-4fad-8238-9da365d8ea59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:20.175160 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a74137-85de-4fad-8238-9da365d8ea59 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:20.175445 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a74137-85de-4fad-8238-9da365d8ea59 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:20.175720 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a74137-85de-4fad-8238-9da365d8ea59 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:20.176374 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a74137-85de-4fad-8238-9da365d8ea59 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:20.177816 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a74137-85de-4fad-8238-9da365d8ea59 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:20.184804 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ee3bc-48e2-4947-a974-45bba3788ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:20.185439 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ee3bc-48e2-4947-a974-45bba3788ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:20.185732 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ee3bc-48e2-4947-a974-45bba3788ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:20.186050 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ee3bc-48e2-4947-a974-45bba3788ec8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:20.250431 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 45/90] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Sep 2 09:13:20 2022] POST /identity/v3/users => generated 289 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:13:20.251282 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a74137-85de-4fad-8238-9da365d8ea59 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:20.260096 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ee3bc-48e2-4947-a974-45bba3788ec8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:20.261673 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ee3bc-48e2-4947-a974-45bba3788ec8 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:20.304443 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:20.360900 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a74137-85de-4fad-8238-9da365d8ea59 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:20.368405 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:20.385929 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 46/91] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:13:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:13:20.386322 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ee3bc-48e2-4947-a974-45bba3788ec8 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:20.401226 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68214cb-44a5-445c-aa34-e843ae65971d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:20.401851 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68214cb-44a5-445c-aa34-e843ae65971d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:20.402140 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68214cb-44a5-445c-aa34-e843ae65971d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:20.403660 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68214cb-44a5-445c-aa34-e843ae65971d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:20.403797 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68214cb-44a5-445c-aa34-e843ae65971d None admin] RBAC: Adding query filter params (role.id=f886b8e8889549e896a8fb3426cce9e1, scope.project.id=79a2aa8010c54ad7ad7b27f08f4de74c, user.id=9cffe229f2f843e394c2248b000ecb37) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:20.404280 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68214cb-44a5-445c-aa34-e843ae65971d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:20.406016 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68214cb-44a5-445c-aa34-e843ae65971d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:20.414998 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 46/92] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:13:20 2022] GET /identity/v3/role_assignments?user.id=9cffe229f2f843e394c2248b000ecb37&scope.project.id=79a2aa8010c54ad7ad7b27f08f4de74c&role.id=f886b8e8889549e896a8fb3426cce9e1 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:20.415751 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68214cb-44a5-445c-aa34-e843ae65971d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:20.518547 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ee3bc-48e2-4947-a974-45bba3788ec8 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:20.565998 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:20.567647 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68214cb-44a5-445c-aa34-e843ae65971d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:20.574988 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc0ca1-4d62-4f7e-84d9-e5e302ff2b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:20.575671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc0ca1-4d62-4f7e-84d9-e5e302ff2b96 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:20.575950 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc0ca1-4d62-4f7e-84d9-e5e302ff2b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:20.576221 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc0ca1-4d62-4f7e-84d9-e5e302ff2b96 None admin] PATH_INFO: `/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:20.577345 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc0ca1-4d62-4f7e-84d9-e5e302ff2b96 None admin] RBAC: Authorizing `identity:get_project(project_id=79a2aa8010c54ad7ad7b27f08f4de74c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:20.579068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc0ca1-4d62-4f7e-84d9-e5e302ff2b96 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:20.580735 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 47/93] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:20 2022] GET /identity/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:20.581580 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc0ca1-4d62-4f7e-84d9-e5e302ff2b96 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:20.652626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a81b4-5024-4e5c-94c9-9dce792e2a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:20.653561 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a81b4-5024-4e5c-94c9-9dce792e2a34 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:20.653852 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a81b4-5024-4e5c-94c9-9dce792e2a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:20.654399 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a81b4-5024-4e5c-94c9-9dce792e2a34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:20.655172 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a81b4-5024-4e5c-94c9-9dce792e2a34 None admin] RBAC: Adding query filter params (role.id=efe6bdd5112340b48b262ceafbc19c04, scope.project.id=79a2aa8010c54ad7ad7b27f08f4de74c, user.id=604723d5b48f4e869bccbe56651aa80b) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:20.655487 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a81b4-5024-4e5c-94c9-9dce792e2a34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:20.657117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a81b4-5024-4e5c-94c9-9dce792e2a34 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:20.669609 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 47/94] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:13:20 2022] GET /identity/v3/role_assignments?user.id=604723d5b48f4e869bccbe56651aa80b&scope.project.id=79a2aa8010c54ad7ad7b27f08f4de74c&role.id=efe6bdd5112340b48b262ceafbc19c04 => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:20.670803 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a81b4-5024-4e5c-94c9-9dce792e2a34 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:20.697204 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc0ca1-4d62-4f7e-84d9-e5e302ff2b96 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:20.734736 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:20.743480 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439081f-f0f4-4ba4-8fe6-dc042f8be8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:20.744144 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439081f-f0f4-4ba4-8fe6-dc042f8be8d6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:20.744427 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439081f-f0f4-4ba4-8fe6-dc042f8be8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:20.744785 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439081f-f0f4-4ba4-8fe6-dc042f8be8d6 None admin] PATH_INFO: `/v3/projects/883aa06e33b5424b90208ef34b521e0d` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:20.746699 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a439081f-f0f4-4ba4-8fe6-dc042f8be8d6 None admin] RBAC: Authorizing `identity:get_project(project_id=883aa06e33b5424b90208ef34b521e0d)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:20.750588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a439081f-f0f4-4ba4-8fe6-dc042f8be8d6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:20.754919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 48/95] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:20 2022] GET /identity/v3/projects/883aa06e33b5424b90208ef34b521e0d => generated 312 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:20.758554 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439081f-f0f4-4ba4-8fe6-dc042f8be8d6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:20.877850 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:20.895352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a81b4-5024-4e5c-94c9-9dce792e2a34 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:20.916555 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439081f-f0f4-4ba4-8fe6-dc042f8be8d6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:20.938897 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf8b8d-8704-47cc-97dc-4cf69e780ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:20.939878 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf8b8d-8704-47cc-97dc-4cf69e780ee5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:20.940388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf8b8d-8704-47cc-97dc-4cf69e780ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:20.940862 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf8b8d-8704-47cc-97dc-4cf69e780ee5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:20.941895 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf8b8d-8704-47cc-97dc-4cf69e780ee5 None admin] RBAC: Adding query filter params (role.id=f2fe9fb31024471e9e2bea2b12715de6, scope.project.id=79a2aa8010c54ad7ad7b27f08f4de74c, user.id=604723d5b48f4e869bccbe56651aa80b) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:20.942455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf8b8d-8704-47cc-97dc-4cf69e780ee5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:20.944163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf8b8d-8704-47cc-97dc-4cf69e780ee5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:20.950854 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 48/96] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:13:20 2022] GET /identity/v3/role_assignments?user.id=604723d5b48f4e869bccbe56651aa80b&scope.project.id=79a2aa8010c54ad7ad7b27f08f4de74c&role.id=f2fe9fb31024471e9e2bea2b12715de6 => generated 259 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:20.957385 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3025aca-2a9b-44bb-a664-75703ccdc711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:20.958010 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3025aca-2a9b-44bb-a664-75703ccdc711 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:20.958297 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3025aca-2a9b-44bb-a664-75703ccdc711 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:20.958649 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3025aca-2a9b-44bb-a664-75703ccdc711 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:20.959415 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3025aca-2a9b-44bb-a664-75703ccdc711 None admin] RBAC: Adding query filter params (role.id=efe6bdd5112340b48b262ceafbc19c04, scope.project.id=883aa06e33b5424b90208ef34b521e0d, user.id=604723d5b48f4e869bccbe56651aa80b) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:20.959725 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3025aca-2a9b-44bb-a664-75703ccdc711 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:20.961164 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3025aca-2a9b-44bb-a664-75703ccdc711 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:20.974902 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 49/97] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:13:20 2022] GET /identity/v3/role_assignments?user.id=604723d5b48f4e869bccbe56651aa80b&scope.project.id=883aa06e33b5424b90208ef34b521e0d&role.id=efe6bdd5112340b48b262ceafbc19c04 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:22.653648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7bede-a3cc-48df-a7f3-e5f8d6dd52a9 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:22.654762 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7bede-a3cc-48df-a7f3-e5f8d6dd52a9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:22.655428 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e7bede-a3cc-48df-a7f3-e5f8d6dd52a9 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:22.657257 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 49/98] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:22.662601 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2624bf-8cb7-49e3-be30-50bff168e2d1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:22.663452 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2624bf-8cb7-49e3-be30-50bff168e2d1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:22.663900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2624bf-8cb7-49e3-be30-50bff168e2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:22.687787 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3e2624bf-8cb7-49e3-be30-50bff168e2d1 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:22.689395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3e2624bf-8cb7-49e3-be30-50bff168e2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:22.777857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 50/99] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:23.182324 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a96f06-3d80-45a2-a8b8-ba99c75a7dac None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:23.182741 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a96f06-3d80-45a2-a8b8-ba99c75a7dac None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:23.182922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a96f06-3d80-45a2-a8b8-ba99c75a7dac None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:23.184904 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 50/100] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:23.190263 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3bb96-efcd-46c6-88e1-5d6707a180f7 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:23.190556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3bb96-efcd-46c6-88e1-5d6707a180f7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:23.190879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3bb96-efcd-46c6-88e1-5d6707a180f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:23.211697 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-77d3bb96-efcd-46c6-88e1-5d6707a180f7 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:23.212899 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-77d3bb96-efcd-46c6-88e1-5d6707a180f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:23.289846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:23.304345 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 51/101] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:23.349098 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a96f06-3d80-45a2-a8b8-ba99c75a7dac None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:23.361162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6524b2-9338-4871-b4df-5818323bc569 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:23.361463 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6524b2-9338-4871-b4df-5818323bc569 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:23.361746 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6524b2-9338-4871-b4df-5818323bc569 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:23.366767 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 52/102] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:23.371975 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d8491-2168-4a1d-9ce7-888f5d0e427d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:23.372258 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d8491-2168-4a1d-9ce7-888f5d0e427d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:23.372534 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d8491-2168-4a1d-9ce7-888f5d0e427d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:23.413415 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ed6d8491-2168-4a1d-9ce7-888f5d0e427d None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:23.417118 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ed6d8491-2168-4a1d-9ce7-888f5d0e427d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:23.563775 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:23.564347 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 53/103] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 09:13:23.569517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a96f06-3d80-45a2-a8b8-ba99c75a7dac None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:23.607071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9af976-753d-44e4-998d-e41ada24c50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:23.608025 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9af976-753d-44e4-998d-e41ada24c50a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:23.608480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9af976-753d-44e4-998d-e41ada24c50a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:23.608927 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9af976-753d-44e4-998d-e41ada24c50a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:23.620577 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9af976-753d-44e4-998d-e41ada24c50a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:23.625722 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2b9af976-753d-44e4-998d-e41ada24c50a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:13:23.625722 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:23.625722 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:23.625722 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:23.625722 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:23.625722 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:23.625722 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:23.625722 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:23.625722 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:23.625722 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:23.625722 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:23.625722 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:23.625722 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:23.625722 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:23.625722 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:23.625722 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:23.626652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:23.626652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:23.626652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:23.626652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:23.626652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:23.626652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:23.626652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:23.626652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:23.626652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:23.626652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:23.626652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:23.626652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:23.626652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:23.626652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:23.626652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:23.626652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:23.627450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:23.627450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:23.627450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:23.627450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:23.627450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:23.627450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:23.627450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:23.627450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:23.627450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:23.627450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:23.627450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:13:23.627450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:13:23.634563 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 51/104] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:13:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:23.636126 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d8491-2168-4a1d-9ce7-888f5d0e427d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:23.798285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf04178-219e-42b3-9035-fef193471065 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:23.799192 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf04178-219e-42b3-9035-fef193471065 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:23.799680 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf04178-219e-42b3-9035-fef193471065 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:23.801186 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 52/105] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:13:23.806064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b92a18-1f47-4d0c-b04a-794eb1c33e1f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:23.806532 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b92a18-1f47-4d0c-b04a-794eb1c33e1f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:23.806998 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b92a18-1f47-4d0c-b04a-794eb1c33e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:23.839337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e6b92a18-1f47-4d0c-b04a-794eb1c33e1f None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:23.841526 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e6b92a18-1f47-4d0c-b04a-794eb1c33e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:23.858399 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d8491-2168-4a1d-9ce7-888f5d0e427d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:23.891660 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:23.906276 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224f843-a15b-4b72-a726-3cc53c68c4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:23.907186 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224f843-a15b-4b72-a726-3cc53c68c4b7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:23.907640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224f843-a15b-4b72-a726-3cc53c68c4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:23.908085 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9224f843-a15b-4b72-a726-3cc53c68c4b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:23.908916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224f843-a15b-4b72-a726-3cc53c68c4b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:23.909410 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224f843-a15b-4b72-a726-3cc53c68c4b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:23.910980 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9224f843-a15b-4b72-a726-3cc53c68c4b7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:23.920927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 54/106] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:23.922881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224f843-a15b-4b72-a726-3cc53c68c4b7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:23.986464 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 53/107] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 09:13:23.990959 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b92a18-1f47-4d0c-b04a-794eb1c33e1f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:24.052580 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9224f843-a15b-4b72-a726-3cc53c68c4b7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:24.087883 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:24.088980 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03694547-d5ec-4cf7-ba34-373917032be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:24.090383 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03694547-d5ec-4cf7-ba34-373917032be2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:24.090611 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03694547-d5ec-4cf7-ba34-373917032be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:24.090914 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03694547-d5ec-4cf7-ba34-373917032be2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:24.100366 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b92a18-1f47-4d0c-b04a-794eb1c33e1f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:24.104481 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03694547-d5ec-4cf7-ba34-373917032be2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:24.111451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-03694547-d5ec-4cf7-ba34-373917032be2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 09:13:24.111451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:24.111451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:24.111451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:24.111451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:24.111451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:24.111451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:24.111451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:24.111451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:24.111451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:24.111451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:24.111451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:24.111451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:24.111451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:24.111451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:24.111451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:24.112300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:24.112300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:24.112300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:24.112300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:24.112300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:24.112300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:24.112300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:24.112300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:24.112300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:24.112300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:24.112300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:24.112300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:24.112300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:24.112300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:24.112300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:24.112300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:24.113046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:24.113046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:24.113046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:24.113046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:24.113046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:24.113046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:24.113046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:24.113046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:24.113046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:24.113046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:24.113046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 09:13:24.113046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:13:24.113046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 55/108] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:24.114167 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03694547-d5ec-4cf7-ba34-373917032be2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:24.134538 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a071a70-4403-4b6e-b122-0f748804b729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:24.135252 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a071a70-4403-4b6e-b122-0f748804b729 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:24.135879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a071a70-4403-4b6e-b122-0f748804b729 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:24.139496 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a071a70-4403-4b6e-b122-0f748804b729 None admin] PATH_INFO: `/v3/users/9cffe229f2f843e394c2248b000ecb37` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:24.140866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a071a70-4403-4b6e-b122-0f748804b729 None admin] RBAC: Authorizing `identity:get_user(user_id=9cffe229f2f843e394c2248b000ecb37)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:24.142356 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a071a70-4403-4b6e-b122-0f748804b729 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:24.144348 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 54/109] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:23 2022] GET /identity/v3/users/9cffe229f2f843e394c2248b000ecb37 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:24.145513 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba07fa-e7df-4e3e-803f-d77b682b7d46 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:24.145799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba07fa-e7df-4e3e-803f-d77b682b7d46 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:24.146075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba07fa-e7df-4e3e-803f-d77b682b7d46 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:24.147498 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 55/110] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:24.152424 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e093c-302c-405c-adb0-4366286791cc None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:24.152717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e093c-302c-405c-adb0-4366286791cc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:24.153001 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e093c-302c-405c-adb0-4366286791cc None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:24.154225 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 56/111] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:24.154996 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e093c-302c-405c-adb0-4366286791cc None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:24.175768 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:24.233513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03694547-d5ec-4cf7-ba34-373917032be2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:24.266028 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e093c-302c-405c-adb0-4366286791cc None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:24.267421 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf8815-d85e-47bc-9017-80256fa8639c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:24.268071 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf8815-d85e-47bc-9017-80256fa8639c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:24.268359 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf8815-d85e-47bc-9017-80256fa8639c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:24.268642 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf8815-d85e-47bc-9017-80256fa8639c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:24.278717 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf8815-d85e-47bc-9017-80256fa8639c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:24.283948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-aecf8815-d85e-47bc-9017-80256fa8639c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 09:13:24.283948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:24.283948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:24.283948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:24.283948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:24.283948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:24.283948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:24.283948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:24.283948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:24.283948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:24.283948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:24.283948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:24.283948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:24.283948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:24.283948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:24.283948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:24.284807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:24.284807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:24.284807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:24.284807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:24.284807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:24.284807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:24.284807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:24.284807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:24.284807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:24.284807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:24.284807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:24.284807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:24.284807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:24.284807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:24.284807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:24.284807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:24.285560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:24.285560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:24.285560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:24.285560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:24.285560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:24.285560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:24.285560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:24.285560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:24.285560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:24.285560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:24.285560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 09:13:24.285560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:13:24.285560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 56/112] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Sep 2 09:13:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:24.286955 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf8815-d85e-47bc-9017-80256fa8639c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:24.301339 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af670139-a0f3-408b-a6b7-592c6bd87430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:24.301950 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af670139-a0f3-408b-a6b7-592c6bd87430 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:24.302228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af670139-a0f3-408b-a6b7-592c6bd87430 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:24.302500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af670139-a0f3-408b-a6b7-592c6bd87430 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:24.303195 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af670139-a0f3-408b-a6b7-592c6bd87430 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:24.303499 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af670139-a0f3-408b-a6b7-592c6bd87430 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:24.304871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af670139-a0f3-408b-a6b7-592c6bd87430 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:24.313689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 57/113] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Sep 2 09:13:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:24.315871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd30a4-6ca9-4ce2-bea0-115f07eec64b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:24.316160 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd30a4-6ca9-4ce2-bea0-115f07eec64b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:24.316435 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd30a4-6ca9-4ce2-bea0-115f07eec64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:24.333384 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-29bd30a4-6ca9-4ce2-bea0-115f07eec64b None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:24.334586 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-29bd30a4-6ca9-4ce2-bea0-115f07eec64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:24.347294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:24.384949 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf8815-d85e-47bc-9017-80256fa8639c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:24.387830 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:24.401022 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 58/114] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:24.402261 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b46f1-b5a2-4c24-a950-40cadff1d26f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:24.402544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b46f1-b5a2-4c24-a950-40cadff1d26f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:24.402856 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b46f1-b5a2-4c24-a950-40cadff1d26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:24.420141 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a79b46f1-b5a2-4c24-a950-40cadff1d26f None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:24.421319 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a79b46f1-b5a2-4c24-a950-40cadff1d26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:24.423373 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb726c97-1ddc-4a48-87d0-9bf6445616cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:24.424908 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb726c97-1ddc-4a48-87d0-9bf6445616cf None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:24.425341 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb726c97-1ddc-4a48-87d0-9bf6445616cf None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:24.425757 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb726c97-1ddc-4a48-87d0-9bf6445616cf None admin] PATH_INFO: `/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:24.427073 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb726c97-1ddc-4a48-87d0-9bf6445616cf None admin] RBAC: Authorizing `identity:get_project(project_id=79a2aa8010c54ad7ad7b27f08f4de74c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:24.429058 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb726c97-1ddc-4a48-87d0-9bf6445616cf None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:24.431526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 57/115] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:24 2022] GET /identity/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:24.433017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb726c97-1ddc-4a48-87d0-9bf6445616cf None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:24.492931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 59/116] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:24.494886 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b46f1-b5a2-4c24-a950-40cadff1d26f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:24.557611 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb726c97-1ddc-4a48-87d0-9bf6445616cf None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:24.586413 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:24.599612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd760c0-1e3c-4561-8139-642e87696dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:24.600700 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd760c0-1e3c-4561-8139-642e87696dce None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:24.601100 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd760c0-1e3c-4561-8139-642e87696dce None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:24.601473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd760c0-1e3c-4561-8139-642e87696dce None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:24.602910 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd760c0-1e3c-4561-8139-642e87696dce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:24.605006 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd760c0-1e3c-4561-8139-642e87696dce None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:24.607990 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 58/117] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:13:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:24.609372 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd760c0-1e3c-4561-8139-642e87696dce None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:24.614165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b46f1-b5a2-4c24-a950-40cadff1d26f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:24.650013 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbb737-b036-4689-8c18-9852fd24089f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:24.650939 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbb737-b036-4689-8c18-9852fd24089f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:24.651404 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbb737-b036-4689-8c18-9852fd24089f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:24.651887 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cbb737-b036-4689-8c18-9852fd24089f None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:24.653296 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cbb737-b036-4689-8c18-9852fd24089f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:24.653805 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cbb737-b036-4689-8c18-9852fd24089f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:24.655553 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cbb737-b036-4689-8c18-9852fd24089f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:24.660510 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:24.664905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 60/118] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:13:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:24.665991 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbb737-b036-4689-8c18-9852fd24089f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:24.726468 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd760c0-1e3c-4561-8139-642e87696dce None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:24.761141 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3da64-bceb-46c7-a47a-254d8e97b84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:24.761803 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3da64-bceb-46c7-a47a-254d8e97b84e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:24.762087 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3da64-bceb-46c7-a47a-254d8e97b84e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:24.762600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3da64-bceb-46c7-a47a-254d8e97b84e None admin] PATH_INFO: `/v3/users/9da1572dde3847a2b938a4c8ed028505` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:24.772289 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cbb737-b036-4689-8c18-9852fd24089f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:24.775618 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3da64-bceb-46c7-a47a-254d8e97b84e None admin] RBAC: Authorizing `identity:get_user(user_id=9da1572dde3847a2b938a4c8ed028505)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:24.777649 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3da64-bceb-46c7-a47a-254d8e97b84e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:24.779519 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 59/119] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:24 2022] GET /identity/v3/users/9da1572dde3847a2b938a4c8ed028505 => generated 289 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:24.780162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3da64-bceb-46c7-a47a-254d8e97b84e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:24.808343 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b13a2-c2fe-4484-8246-d6a734ec1473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:24.809007 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b13a2-c2fe-4484-8246-d6a734ec1473 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:24.809347 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b13a2-c2fe-4484-8246-d6a734ec1473 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:24.809623 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b13a2-c2fe-4484-8246-d6a734ec1473 None admin] PATH_INFO: `/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c/users/9cffe229f2f843e394c2248b000ecb37/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:24.811613 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957b13a2-c2fe-4484-8246-d6a734ec1473 None admin] RBAC: Authorizing `identity:create_grant(project_id=79a2aa8010c54ad7ad7b27f08f4de74c, user_id=9cffe229f2f843e394c2248b000ecb37, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:24.813058 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957b13a2-c2fe-4484-8246-d6a734ec1473 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:24.863766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:24.868078 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 61/120] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Sep 2 09:13:24 2022] PUT /identity/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c/users/9cffe229f2f843e394c2248b000ecb37/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:13:24.874885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b13a2-c2fe-4484-8246-d6a734ec1473 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:24.886299 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:24.891820 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3da64-bceb-46c7-a47a-254d8e97b84e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:24.930753 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f52e4-bf19-4780-bbd5-43163cdd1127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:24.931443 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f52e4-bf19-4780-bbd5-43163cdd1127 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:24.931727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f52e4-bf19-4780-bbd5-43163cdd1127 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:24.932024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f52e4-bf19-4780-bbd5-43163cdd1127 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:24.932696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989f52e4-bf19-4780-bbd5-43163cdd1127 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:24.934250 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989f52e4-bf19-4780-bbd5-43163cdd1127 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:24.991450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b13a2-c2fe-4484-8246-d6a734ec1473 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:25.028393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32379074-2491-4e91-9d87-c68ea245dec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:25.029312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32379074-2491-4e91-9d87-c68ea245dec6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:25.029608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32379074-2491-4e91-9d87-c68ea245dec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:25.029892 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32379074-2491-4e91-9d87-c68ea245dec6 None admin] PATH_INFO: `/v3/users/604723d5b48f4e869bccbe56651aa80b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:25.031220 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32379074-2491-4e91-9d87-c68ea245dec6 None admin] RBAC: Authorizing `identity:get_user(user_id=604723d5b48f4e869bccbe56651aa80b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:25.032674 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32379074-2491-4e91-9d87-c68ea245dec6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:25.034319 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 62/121] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:24 2022] GET /identity/v3/users/604723d5b48f4e869bccbe56651aa80b => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:25.035182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32379074-2491-4e91-9d87-c68ea245dec6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:25.047608 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 60/122] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Sep 2 09:13:24 2022] POST /identity/v3/projects => generated 302 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:13:25.047775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f52e4-bf19-4780-bbd5-43163cdd1127 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:25.133287 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:25.163685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f52e4-bf19-4780-bbd5-43163cdd1127 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:25.188528 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32379074-2491-4e91-9d87-c68ea245dec6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:25.206849 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda2308-96be-477e-a1ac-dacb520862d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:25.208190 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda2308-96be-477e-a1ac-dacb520862d7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:25.208482 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda2308-96be-477e-a1ac-dacb520862d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:25.208769 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda2308-96be-477e-a1ac-dacb520862d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:25.224036 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda2308-96be-477e-a1ac-dacb520862d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:25.229301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-8eda2308-96be-477e-a1ac-dacb520862d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:25.229301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:25.229301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:25.229301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:25.229301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:25.229301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:25.229301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:25.229301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:25.229301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:25.229301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:25.229301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:25.229301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:25.229301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:25.229301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:25.229301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:25.229301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:25.230431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:25.230431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:25.230431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:25.230431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:25.230431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:25.230431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:25.230431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:25.230431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:25.230431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:25.230431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:25.230431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:25.230431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:25.230431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:25.230431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:25.230431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:25.230431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:25.231526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:25.231526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:25.231526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:25.231526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:25.231526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:25.231526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:25.231526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:25.231526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:25.231526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:25.231526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:25.231526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:25.231526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:13:25.231526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 61/123] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:25.231526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda2308-96be-477e-a1ac-dacb520862d7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:25.236796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd152a27-9bf4-4627-8bfb-f31955d2ebe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:25.237625 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd152a27-9bf4-4627-8bfb-f31955d2ebe1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:25.238103 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd152a27-9bf4-4627-8bfb-f31955d2ebe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:25.238520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd152a27-9bf4-4627-8bfb-f31955d2ebe1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:25.247569 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:25.251352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd152a27-9bf4-4627-8bfb-f31955d2ebe1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:25.257030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-dd152a27-9bf4-4627-8bfb-f31955d2ebe1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:25.257030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:25.257030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:25.257030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:25.257030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:25.257030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:25.257030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:25.257030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:25.257030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:25.257030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:25.257030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:25.257030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:25.257030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:25.257030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:25.257030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:25.257030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:25.261200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:25.261200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:25.261200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:25.261200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:25.261200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:25.261200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:25.261200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:25.261200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:25.261200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:25.261200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:25.261200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:25.261200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:25.261200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:25.261200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:25.261200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:25.261200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:25.262651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:25.262651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:25.262651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:25.262651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:25.262651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:25.262651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:25.262651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:25.262651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:25.262651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:25.262651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:25.262651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:25.262651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:13:25.262651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 63/124] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:25.262651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd152a27-9bf4-4627-8bfb-f31955d2ebe1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:25.352131 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda2308-96be-477e-a1ac-dacb520862d7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:25.370302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd152a27-9bf4-4627-8bfb-f31955d2ebe1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:25.397370 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecae289-6e98-43c6-a157-066fde820f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:25.398514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecae289-6e98-43c6-a157-066fde820f4c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:25.399192 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecae289-6e98-43c6-a157-066fde820f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:25.399616 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecae289-6e98-43c6-a157-066fde820f4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:25.400649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:25.408772 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0cf3c-8e7a-477a-a8f1-7eb8bbe79995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:25.409523 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0cf3c-8e7a-477a-a8f1-7eb8bbe79995 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:25.409867 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0cf3c-8e7a-477a-a8f1-7eb8bbe79995 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:25.410207 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca0cf3c-8e7a-477a-a8f1-7eb8bbe79995 None admin] PATH_INFO: `/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:25.414444 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca0cf3c-8e7a-477a-a8f1-7eb8bbe79995 None admin] RBAC: Authorizing `identity:get_project(project_id=79a2aa8010c54ad7ad7b27f08f4de74c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:25.414444 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca0cf3c-8e7a-477a-a8f1-7eb8bbe79995 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:25.415155 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 64/125] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:25 2022] GET /identity/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:25.417225 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0cf3c-8e7a-477a-a8f1-7eb8bbe79995 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:25.497901 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecae289-6e98-43c6-a157-066fde820f4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:25.499639 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecae289-6e98-43c6-a157-066fde820f4c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:25.516722 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:25.528322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca0cf3c-8e7a-477a-a8f1-7eb8bbe79995 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:25.571683 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99034668-fc61-473d-85d3-e202a7773a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:25.573250 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99034668-fc61-473d-85d3-e202a7773a34 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:25.573677 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99034668-fc61-473d-85d3-e202a7773a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:25.574665 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99034668-fc61-473d-85d3-e202a7773a34 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:25.576191 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99034668-fc61-473d-85d3-e202a7773a34 None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:25.576690 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99034668-fc61-473d-85d3-e202a7773a34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:25.578973 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99034668-fc61-473d-85d3-e202a7773a34 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:25.590549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 65/126] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Sep 2 09:13:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:25.591662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99034668-fc61-473d-85d3-e202a7773a34 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:25.623631 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 62/127] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:13:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:13:25.624837 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecae289-6e98-43c6-a157-066fde820f4c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:25.666218 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:25.727327 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99034668-fc61-473d-85d3-e202a7773a34 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:25.729855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecae289-6e98-43c6-a157-066fde820f4c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:25.743232 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:25.769002 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e25d07-5512-41d0-8745-08467ee00ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:25.769862 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e25d07-5512-41d0-8745-08467ee00ba7 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:25.770248 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e25d07-5512-41d0-8745-08467ee00ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:25.770628 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e25d07-5512-41d0-8745-08467ee00ba7 None admin] PATH_INFO: `/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c/users/604723d5b48f4e869bccbe56651aa80b/roles/f2fe9fb31024471e9e2bea2b12715de6` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:25.772455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701cc20-79b7-4099-918a-b878038fbe35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:25.773118 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e25d07-5512-41d0-8745-08467ee00ba7 None admin] RBAC: Authorizing `identity:create_grant(project_id=79a2aa8010c54ad7ad7b27f08f4de74c, user_id=604723d5b48f4e869bccbe56651aa80b, role_id=f2fe9fb31024471e9e2bea2b12715de6)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:25.773382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701cc20-79b7-4099-918a-b878038fbe35 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:25.773799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701cc20-79b7-4099-918a-b878038fbe35 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:25.774226 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701cc20-79b7-4099-918a-b878038fbe35 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:25.775064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701cc20-79b7-4099-918a-b878038fbe35 None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:25.775204 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e25d07-5512-41d0-8745-08467ee00ba7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:25.775649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701cc20-79b7-4099-918a-b878038fbe35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:25.777698 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701cc20-79b7-4099-918a-b878038fbe35 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:25.788075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 66/128] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Sep 2 09:13:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:25.789085 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701cc20-79b7-4099-918a-b878038fbe35 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:25.884091 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 63/129] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Sep 2 09:13:25 2022] PUT /identity/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c/users/604723d5b48f4e869bccbe56651aa80b/roles/f2fe9fb31024471e9e2bea2b12715de6 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:13:25.885448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e25d07-5512-41d0-8745-08467ee00ba7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:25.894266 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701cc20-79b7-4099-918a-b878038fbe35 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:25.918845 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:25.936754 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6fa839-17d4-434e-93ce-eca01af6a69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:25.937408 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6fa839-17d4-434e-93ce-eca01af6a69d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:25.937689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6fa839-17d4-434e-93ce-eca01af6a69d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:25.937960 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6fa839-17d4-434e-93ce-eca01af6a69d None admin] PATH_INFO: `/v3/users/604723d5b48f4e869bccbe56651aa80b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:25.940150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6fa839-17d4-434e-93ce-eca01af6a69d None admin] RBAC: Authorizing `identity:get_user(user_id=604723d5b48f4e869bccbe56651aa80b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:25.940996 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6fa839-17d4-434e-93ce-eca01af6a69d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:25.943487 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 67/130] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:25 2022] GET /identity/v3/users/604723d5b48f4e869bccbe56651aa80b => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:25.944253 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6fa839-17d4-434e-93ce-eca01af6a69d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:26.080181 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6fa839-17d4-434e-93ce-eca01af6a69d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:26.097729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e25d07-5512-41d0-8745-08467ee00ba7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:26.125418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371dc67-17ff-4fea-9472-781d90031a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:26.126069 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371dc67-17ff-4fea-9472-781d90031a8b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:26.126350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371dc67-17ff-4fea-9472-781d90031a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:26.126624 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371dc67-17ff-4fea-9472-781d90031a8b None admin] PATH_INFO: `/v3/users/604723d5b48f4e869bccbe56651aa80b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:26.130695 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371dc67-17ff-4fea-9472-781d90031a8b None admin] RBAC: Authorizing `identity:get_user(user_id=604723d5b48f4e869bccbe56651aa80b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:26.132170 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6371dc67-17ff-4fea-9472-781d90031a8b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:26.138591 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 68/131] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:25 2022] GET /identity/v3/users/604723d5b48f4e869bccbe56651aa80b => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:26.139243 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371dc67-17ff-4fea-9472-781d90031a8b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:26.173987 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985678a7-975a-4324-bbc2-953c75f52961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:26.175045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985678a7-975a-4324-bbc2-953c75f52961 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:26.175529 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985678a7-975a-4324-bbc2-953c75f52961 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:26.176003 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985678a7-975a-4324-bbc2-953c75f52961 None admin] PATH_INFO: `/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:26.177365 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985678a7-975a-4324-bbc2-953c75f52961 None admin] RBAC: Authorizing `identity:get_project(project_id=79a2aa8010c54ad7ad7b27f08f4de74c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:26.179219 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985678a7-975a-4324-bbc2-953c75f52961 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:26.181446 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 64/132] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:25 2022] GET /identity/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c => generated 298 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:26.182544 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985678a7-975a-4324-bbc2-953c75f52961 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:26.223786 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:26.239089 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:26.277633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371dc67-17ff-4fea-9472-781d90031a8b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:26.318760 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7dd02-e468-4a29-8df0-b5898378a50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:26.320126 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7dd02-e468-4a29-8df0-b5898378a50f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:26.320681 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7dd02-e468-4a29-8df0-b5898378a50f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:26.321221 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf7dd02-e468-4a29-8df0-b5898378a50f None admin] PATH_INFO: `/v3/projects/883aa06e33b5424b90208ef34b521e0d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:26.323342 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7dd02-e468-4a29-8df0-b5898378a50f None admin] RBAC: Authorizing `identity:get_project(project_id=883aa06e33b5424b90208ef34b521e0d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:26.324991 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf7dd02-e468-4a29-8df0-b5898378a50f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:26.326739 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 69/133] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:26 2022] GET /identity/v3/projects/883aa06e33b5424b90208ef34b521e0d => generated 312 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:26.327819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7dd02-e468-4a29-8df0-b5898378a50f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:26.348663 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985678a7-975a-4324-bbc2-953c75f52961 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:26.412418 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad4495-6e97-4e18-9d83-303a03f41452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:26.413074 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad4495-6e97-4e18-9d83-303a03f41452 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:26.413360 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad4495-6e97-4e18-9d83-303a03f41452 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:26.413645 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad4495-6e97-4e18-9d83-303a03f41452 None admin] PATH_INFO: `/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:26.419572 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad4495-6e97-4e18-9d83-303a03f41452 None admin] RBAC: Authorizing `identity:get_project(project_id=79a2aa8010c54ad7ad7b27f08f4de74c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:26.421380 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad4495-6e97-4e18-9d83-303a03f41452 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:26.423527 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 65/134] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:26 2022] GET /identity/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:26.424673 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad4495-6e97-4e18-9d83-303a03f41452 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:26.439890 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf7dd02-e468-4a29-8df0-b5898378a50f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:26.471095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:26.475513 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d921f93-693d-4c09-9b93-615414925cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:26.476188 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d921f93-693d-4c09-9b93-615414925cfb None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:26.476473 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d921f93-693d-4c09-9b93-615414925cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:26.476750 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d921f93-693d-4c09-9b93-615414925cfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:26.477639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d921f93-693d-4c09-9b93-615414925cfb None admin] RBAC: Adding query filter params (role.id=b381d11646e54d1197676600202a6144, scope.project.id=79a2aa8010c54ad7ad7b27f08f4de74c, user.id=9da1572dde3847a2b938a4c8ed028505) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:26.477953 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d921f93-693d-4c09-9b93-615414925cfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:26.479411 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d921f93-693d-4c09-9b93-615414925cfb None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:26.485548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 70/135] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:13:26 2022] GET /identity/v3/role_assignments?user.id=9da1572dde3847a2b938a4c8ed028505&scope.project.id=79a2aa8010c54ad7ad7b27f08f4de74c&role.id=b381d11646e54d1197676600202a6144 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:26.486444 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d921f93-693d-4c09-9b93-615414925cfb None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:26.574451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad4495-6e97-4e18-9d83-303a03f41452 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:26.591395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:26.616591 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d921f93-693d-4c09-9b93-615414925cfb None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:26.644899 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f2f347-f405-44ca-81bb-b5fb87faa379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:26.645585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2f347-f405-44ca-81bb-b5fb87faa379 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:26.645876 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2f347-f405-44ca-81bb-b5fb87faa379 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:26.646158 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f2f347-f405-44ca-81bb-b5fb87faa379 None admin] PATH_INFO: `/v3/projects/883aa06e33b5424b90208ef34b521e0d/users/604723d5b48f4e869bccbe56651aa80b/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:26.652785 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f2f347-f405-44ca-81bb-b5fb87faa379 None admin] RBAC: Authorizing `identity:create_grant(project_id=883aa06e33b5424b90208ef34b521e0d, user_id=604723d5b48f4e869bccbe56651aa80b, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:26.654300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f2f347-f405-44ca-81bb-b5fb87faa379 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:26.664601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e4f70f-2415-49ae-9831-abd3a5cbd5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:26.665349 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4f70f-2415-49ae-9831-abd3a5cbd5d7 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:26.665645 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4f70f-2415-49ae-9831-abd3a5cbd5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:26.665929 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e4f70f-2415-49ae-9831-abd3a5cbd5d7 None admin] PATH_INFO: `/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c/users/604723d5b48f4e869bccbe56651aa80b/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:26.669826 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e4f70f-2415-49ae-9831-abd3a5cbd5d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=79a2aa8010c54ad7ad7b27f08f4de74c, user_id=604723d5b48f4e869bccbe56651aa80b, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:26.671398 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e4f70f-2415-49ae-9831-abd3a5cbd5d7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:26.731651 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 66/136] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Sep 2 09:13:26 2022] PUT /identity/v3/projects/883aa06e33b5424b90208ef34b521e0d/users/604723d5b48f4e869bccbe56651aa80b/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:13:26.760983 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 71/137] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Sep 2 09:13:26 2022] PUT /identity/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c/users/604723d5b48f4e869bccbe56651aa80b/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:13:26.960857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb6047-26bc-41a1-a16b-3fc7db03edf7 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:26.961187 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb6047-26bc-41a1-a16b-3fc7db03edf7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:26.961423 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb6047-26bc-41a1-a16b-3fc7db03edf7 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:26.963051 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 67/138] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:26.969578 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f0006-f96d-4379-94ab-8e64ef4de399 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:26.970093 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f0006-f96d-4379-94ab-8e64ef4de399 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:26.970093 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f0006-f96d-4379-94ab-8e64ef4de399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:26.994688 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b05f0006-f96d-4379-94ab-8e64ef4de399 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:26.996008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b05f0006-f96d-4379-94ab-8e64ef4de399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:27.052725 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:27.080538 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 72/139] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:27.402618 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb6047-26bc-41a1-a16b-3fc7db03edf7 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:27.483776 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba0406-0d6d-4938-ac35-be51af9e17d9 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:27.484087 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba0406-0d6d-4938-ac35-be51af9e17d9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:27.484357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba0406-0d6d-4938-ac35-be51af9e17d9 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:27.485861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 73/140] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:27.504503 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0815629-6e08-4d97-b7ea-ba82102877ed None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:27.504797 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0815629-6e08-4d97-b7ea-ba82102877ed None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:27.505056 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0815629-6e08-4d97-b7ea-ba82102877ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:27.540018 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f0815629-6e08-4d97-b7ea-ba82102877ed None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:27.541249 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f0815629-6e08-4d97-b7ea-ba82102877ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:27.560808 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb6047-26bc-41a1-a16b-3fc7db03edf7 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:27.598100 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08546eac-af6a-4230-87bd-e7fe3b525b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:27.598759 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08546eac-af6a-4230-87bd-e7fe3b525b69 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:27.599046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08546eac-af6a-4230-87bd-e7fe3b525b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:27.599318 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08546eac-af6a-4230-87bd-e7fe3b525b69 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:27.610938 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08546eac-af6a-4230-87bd-e7fe3b525b69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:27.616225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-08546eac-af6a-4230-87bd-e7fe3b525b69 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:13:27.616225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:27.616225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:27.616225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:27.616225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:27.616225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:27.616225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:27.616225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:27.616225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:27.616225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:27.616225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:27.616225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:27.616225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:27.616225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:27.616225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:27.616225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:27.617162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:27.617162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:27.617162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:27.617162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:27.617162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:27.617162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:27.617162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:27.617162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:27.617162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:27.617162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:27.617162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:27.617162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:27.617162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:27.617162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:27.617162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:27.617162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:27.617919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:27.617919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:27.617919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:27.617919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:27.617919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:27.617919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:27.617919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:27.617919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:27.617919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:27.617919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:27.617919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:13:27.617919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:13:27.617919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 68/141] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:13:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:27.627995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08546eac-af6a-4230-87bd-e7fe3b525b69 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:27.676544 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:27.677020 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 74/142] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 09:13:27.768425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08546eac-af6a-4230-87bd-e7fe3b525b69 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:27.803642 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d53e1a-878e-4bc4-9f08-fbf65ebfe250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:27.804262 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d53e1a-878e-4bc4-9f08-fbf65ebfe250 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:27.804551 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d53e1a-878e-4bc4-9f08-fbf65ebfe250 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:27.804831 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d53e1a-878e-4bc4-9f08-fbf65ebfe250 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:27.805506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d53e1a-878e-4bc4-9f08-fbf65ebfe250 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:27.805817 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d53e1a-878e-4bc4-9f08-fbf65ebfe250 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:27.807263 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d53e1a-878e-4bc4-9f08-fbf65ebfe250 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:27.816184 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 69/143] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:27.824060 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0815629-6e08-4d97-b7ea-ba82102877ed None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:27.955626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0815629-6e08-4d97-b7ea-ba82102877ed None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:27.965785 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:27.998340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2120e49-dd30-44d5-9bff-9d07ce12b784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:28.000022 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2120e49-dd30-44d5-9bff-9d07ce12b784 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:28.000576 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2120e49-dd30-44d5-9bff-9d07ce12b784 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:28.001199 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2120e49-dd30-44d5-9bff-9d07ce12b784 None admin] PATH_INFO: `/v3/users/9cffe229f2f843e394c2248b000ecb37` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:28.002665 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2120e49-dd30-44d5-9bff-9d07ce12b784 None admin] RBAC: Authorizing `identity:get_user(user_id=9cffe229f2f843e394c2248b000ecb37)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:28.004697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2120e49-dd30-44d5-9bff-9d07ce12b784 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:28.006557 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 75/144] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:27 2022] GET /identity/v3/users/9cffe229f2f843e394c2248b000ecb37 => generated 247 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:28.013145 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d53e1a-878e-4bc4-9f08-fbf65ebfe250 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:28.125556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:28.141397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d53e1a-878e-4bc4-9f08-fbf65ebfe250 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:28.188573 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d504d-1a18-4dd5-a141-0a5299dd88f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:28.189195 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d504d-1a18-4dd5-a141-0a5299dd88f7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:28.189484 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d504d-1a18-4dd5-a141-0a5299dd88f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:28.189769 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d504d-1a18-4dd5-a141-0a5299dd88f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:28.239896 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2120e49-dd30-44d5-9bff-9d07ce12b784 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:28.282011 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96d504d-1a18-4dd5-a141-0a5299dd88f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:28.283517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96d504d-1a18-4dd5-a141-0a5299dd88f7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:28.402745 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2120e49-dd30-44d5-9bff-9d07ce12b784 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:28.441221 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f6848-3468-442a-b055-1cd750fd682e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:28.441885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f6848-3468-442a-b055-1cd750fd682e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:28.442177 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f6848-3468-442a-b055-1cd750fd682e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:28.442456 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f6848-3468-442a-b055-1cd750fd682e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:28.443766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9f6848-3468-442a-b055-1cd750fd682e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:28.445415 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9f6848-3468-442a-b055-1cd750fd682e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:28.447210 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 76/145] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:13:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:28.455861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f6848-3468-442a-b055-1cd750fd682e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:28.531157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:28.566681 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:28.590120 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f6848-3468-442a-b055-1cd750fd682e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:28.612673 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 70/146] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:13:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:13:28.624116 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d504d-1a18-4dd5-a141-0a5299dd88f7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:28.655404 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a4cab3-7dd9-453b-bc12-4f4e5ddfe343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:28.656032 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a4cab3-7dd9-453b-bc12-4f4e5ddfe343 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:28.656314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a4cab3-7dd9-453b-bc12-4f4e5ddfe343 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:28.656589 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a4cab3-7dd9-453b-bc12-4f4e5ddfe343 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:28.657266 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a4cab3-7dd9-453b-bc12-4f4e5ddfe343 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:28.658739 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a4cab3-7dd9-453b-bc12-4f4e5ddfe343 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:28.766552 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d504d-1a18-4dd5-a141-0a5299dd88f7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:28.799621 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 77/147] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Sep 2 09:13:28 2022] POST /identity/v3/users => generated 283 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:13:28.800995 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53ade5-38a6-4025-9a08-5d5bbb2b0e45 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:28.801287 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53ade5-38a6-4025-9a08-5d5bbb2b0e45 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:28.801564 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a53ade5-38a6-4025-9a08-5d5bbb2b0e45 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:28.802826 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 78/148] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:28.816753 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353b71f-c3f3-4174-a597-c9726999f32a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:28.817069 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353b71f-c3f3-4174-a597-c9726999f32a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:28.817299 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353b71f-c3f3-4174-a597-c9726999f32a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:28.832313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76d014-3584-4702-8f80-f842f660825e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:28.832966 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76d014-3584-4702-8f80-f842f660825e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:28.833246 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76d014-3584-4702-8f80-f842f660825e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:28.833518 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a76d014-3584-4702-8f80-f842f660825e None admin] PATH_INFO: `/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:28.835478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a76d014-3584-4702-8f80-f842f660825e None admin] RBAC: Authorizing `identity:get_project(project_id=79a2aa8010c54ad7ad7b27f08f4de74c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:28.837330 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a76d014-3584-4702-8f80-f842f660825e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:28.839619 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 71/149] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:28 2022] GET /identity/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:28.845762 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76d014-3584-4702-8f80-f842f660825e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:28.847932 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a353b71f-c3f3-4174-a597-c9726999f32a None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:28.849955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a353b71f-c3f3-4174-a597-c9726999f32a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:28.971140 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:28.976724 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 79/150] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 09:13:29.054055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a76d014-3584-4702-8f80-f842f660825e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:29.089950 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a712fd-c584-4b22-a24d-0e015721c7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:29.090564 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a712fd-c584-4b22-a24d-0e015721c7ff None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:29.090875 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a712fd-c584-4b22-a24d-0e015721c7ff None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:29.091149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a712fd-c584-4b22-a24d-0e015721c7ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:29.091878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a712fd-c584-4b22-a24d-0e015721c7ff None admin] RBAC: Adding query filter params (role.id=f886b8e8889549e896a8fb3426cce9e1, scope.project.id=79a2aa8010c54ad7ad7b27f08f4de74c, user.id=9cffe229f2f843e394c2248b000ecb37) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:29.092182 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a712fd-c584-4b22-a24d-0e015721c7ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:29.093623 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a712fd-c584-4b22-a24d-0e015721c7ff None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:29.099532 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 72/151] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:13:28 2022] GET /identity/v3/role_assignments?user.id=9cffe229f2f843e394c2248b000ecb37&scope.project.id=79a2aa8010c54ad7ad7b27f08f4de74c&role.id=f886b8e8889549e896a8fb3426cce9e1 => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:13:29.293721 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9393bf-ee0b-4a7e-9ea9-beebb4beaad0 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:29.294221 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9393bf-ee0b-4a7e-9ea9-beebb4beaad0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:29.294292 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9393bf-ee0b-4a7e-9ea9-beebb4beaad0 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:29.295761 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 80/152] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:29.301224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3cfc6-9bdd-4a5e-b0f1-bf51d6d9b394 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:29.301499 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3cfc6-9bdd-4a5e-b0f1-bf51d6d9b394 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:29.301769 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3cfc6-9bdd-4a5e-b0f1-bf51d6d9b394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:29.332813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3ba3cfc6-9bdd-4a5e-b0f1-bf51d6d9b394 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:29.333928 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3ba3cfc6-9bdd-4a5e-b0f1-bf51d6d9b394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:29.414655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 73/153] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:29.446137 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9393bf-ee0b-4a7e-9ea9-beebb4beaad0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:29.479881 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:29.565473 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9393bf-ee0b-4a7e-9ea9-beebb4beaad0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:29.605634 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b7c87-3a89-4cf7-9638-0fc9a9683971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:29.606273 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b7c87-3a89-4cf7-9638-0fc9a9683971 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:29.607254 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b7c87-3a89-4cf7-9638-0fc9a9683971 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:29.607254 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b7c87-3a89-4cf7-9638-0fc9a9683971 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:29.618602 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8b7c87-3a89-4cf7-9638-0fc9a9683971 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:29.624347 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-3a8b7c87-3a89-4cf7-9638-0fc9a9683971 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 09:13:29.624347 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:29.624347 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:29.624347 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:29.624347 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:29.624347 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:29.624347 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:29.624347 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:29.624347 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:29.624347 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:29.624347 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:29.624347 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:29.624347 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:29.624347 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:29.624347 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:29.624347 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:29.625181 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:29.625181 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:29.625181 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:29.625181 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:29.625181 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:29.625181 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:29.625181 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:29.625181 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:29.625181 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:29.625181 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:29.625181 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:29.625181 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:29.625181 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:29.625181 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:29.625181 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:29.625181 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:29.625941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:29.625941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:29.625941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:29.625941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:29.625941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:29.625941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:29.625941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:29.625941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:29.625941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:29.625941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:29.625941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 09:13:29.625941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:13:29.625941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 81/154] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Sep 2 09:13:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:29.631763 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba3cfc6-9bdd-4a5e-b0f1-bf51d6d9b394 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:29.702245 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:29.747299 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94379b6c-315c-4a8d-b2d0-a27bffa6121b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:29.747621 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94379b6c-315c-4a8d-b2d0-a27bffa6121b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:29.747847 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94379b6c-315c-4a8d-b2d0-a27bffa6121b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:29.749615 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 82/155] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:29.755891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a75b1-ba0f-4038-a39a-00f55df96877 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:29.756174 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a75b1-ba0f-4038-a39a-00f55df96877 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:29.756451 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a75b1-ba0f-4038-a39a-00f55df96877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:29.766113 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba3cfc6-9bdd-4a5e-b0f1-bf51d6d9b394 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:29.784756 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-744a75b1-ba0f-4038-a39a-00f55df96877 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:29.786179 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-744a75b1-ba0f-4038-a39a-00f55df96877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:29.808478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ffb3a-8d19-4604-bb8b-2d1d324fc242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:29.809097 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ffb3a-8d19-4604-bb8b-2d1d324fc242 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:29.809378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ffb3a-8d19-4604-bb8b-2d1d324fc242 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:29.809649 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ffb3a-8d19-4604-bb8b-2d1d324fc242 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:29.810314 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ffb3a-8d19-4604-bb8b-2d1d324fc242 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:29.810617 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ffb3a-8d19-4604-bb8b-2d1d324fc242 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:29.812027 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ffb3a-8d19-4604-bb8b-2d1d324fc242 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:29.820405 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 74/156] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:13:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:29.821614 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e274b-4cc3-4a5c-bd84-07f384a3746e None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:29.821896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e274b-4cc3-4a5c-bd84-07f384a3746e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:29.822167 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e274b-4cc3-4a5c-bd84-07f384a3746e None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:29.825306 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 75/157] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:29.829595 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e274b-4cc3-4a5c-bd84-07f384a3746e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:29.879755 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 83/158] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 09:13:29.881967 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a75b1-ba0f-4038-a39a-00f55df96877 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:29.909117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:29.961442 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e274b-4cc3-4a5c-bd84-07f384a3746e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:29.991076 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:30.003775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f027631-533b-4a8e-a748-1ae6e7820ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:30.004845 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f027631-533b-4a8e-a748-1ae6e7820ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:30.005308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f027631-533b-4a8e-a748-1ae6e7820ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:30.005739 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a75b1-ba0f-4038-a39a-00f55df96877 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:30.006000 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f027631-533b-4a8e-a748-1ae6e7820ee7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:30.021859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f027631-533b-4a8e-a748-1ae6e7820ee7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:30.027678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-3f027631-533b-4a8e-a748-1ae6e7820ee7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 09:13:30.027678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:30.027678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:30.027678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:30.027678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:30.027678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:30.027678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:30.027678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:30.027678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:30.027678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:30.027678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:30.027678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:30.027678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:30.027678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:30.027678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:30.027678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:30.028585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:30.028585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:30.028585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:30.028585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:30.028585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:30.028585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:30.028585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:30.028585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:30.028585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:30.028585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:30.028585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:30.028585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:30.028585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:30.028585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:30.028585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:30.028585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:30.029420 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:30.029420 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:30.029420 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:30.029420 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:30.029420 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:30.029420 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:30.029420 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:30.029420 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:30.029420 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:30.029420 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:30.029420 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 09:13:30.029420 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:13:30.031475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 76/159] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:30.032934 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d32b9-bc4a-451e-9798-512c14f99075 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:30.033477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d32b9-bc4a-451e-9798-512c14f99075 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:30.033894 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d32b9-bc4a-451e-9798-512c14f99075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:30.045288 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4832f0-a09c-4f2d-bc10-7f151de08413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:30.045934 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4832f0-a09c-4f2d-bc10-7f151de08413 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:30.046211 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4832f0-a09c-4f2d-bc10-7f151de08413 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:30.046480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4832f0-a09c-4f2d-bc10-7f151de08413 None admin] PATH_INFO: `/v3/users/604723d5b48f4e869bccbe56651aa80b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:30.048138 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4832f0-a09c-4f2d-bc10-7f151de08413 None admin] RBAC: Authorizing `identity:get_user(user_id=604723d5b48f4e869bccbe56651aa80b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:30.050106 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4832f0-a09c-4f2d-bc10-7f151de08413 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:30.051807 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 84/160] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:29 2022] GET /identity/v3/users/604723d5b48f4e869bccbe56651aa80b => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:30.052628 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4832f0-a09c-4f2d-bc10-7f151de08413 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:30.055902 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-420d32b9-bc4a-451e-9798-512c14f99075 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:30.057367 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-420d32b9-bc4a-451e-9798-512c14f99075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:30.147838 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 77/161] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:30.148913 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d32b9-bc4a-451e-9798-512c14f99075 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:30.166606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:30.190690 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4832f0-a09c-4f2d-bc10-7f151de08413 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:30.241443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143b286-72ec-4972-aa21-6b3b2de704e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:30.242061 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143b286-72ec-4972-aa21-6b3b2de704e6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:30.242336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143b286-72ec-4972-aa21-6b3b2de704e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:30.242605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e143b286-72ec-4972-aa21-6b3b2de704e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:30.243342 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e143b286-72ec-4972-aa21-6b3b2de704e6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:30.243649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e143b286-72ec-4972-aa21-6b3b2de704e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:30.245142 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e143b286-72ec-4972-aa21-6b3b2de704e6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:30.255417 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 85/162] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Sep 2 09:13:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:30.256233 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143b286-72ec-4972-aa21-6b3b2de704e6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:30.320826 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d32b9-bc4a-451e-9798-512c14f99075 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:30.322636 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:30.389305 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be1dbe-d12b-4dbf-89c9-20b7087f4788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:30.389923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be1dbe-d12b-4dbf-89c9-20b7087f4788 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:30.390198 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be1dbe-d12b-4dbf-89c9-20b7087f4788 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:30.390468 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be1dbe-d12b-4dbf-89c9-20b7087f4788 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:30.406891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e143b286-72ec-4972-aa21-6b3b2de704e6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:30.466278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccbc3ae-32fb-46ac-9185-0944af140acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:30.466936 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbc3ae-32fb-46ac-9185-0944af140acf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:30.467217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbc3ae-32fb-46ac-9185-0944af140acf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:30.467487 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccbc3ae-32fb-46ac-9185-0944af140acf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:30.485942 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccbc3ae-32fb-46ac-9185-0944af140acf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:30.491182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-9ccbc3ae-32fb-46ac-9185-0944af140acf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:30.491182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:30.491182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:30.491182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:30.491182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:30.491182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:30.491182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:30.491182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:30.491182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:30.491182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:30.491182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:30.491182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:30.491182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:30.491182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:30.491182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:30.491182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:30.492083 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:30.492083 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:30.492083 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:30.492083 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:30.492083 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:30.492083 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:30.492083 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:30.492083 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:30.492083 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:30.492083 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:30.492083 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:30.492083 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:30.492083 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:30.492083 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:30.492083 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:30.492083 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:30.492928 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:30.492928 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:30.492928 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:30.492928 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:30.492928 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:30.492928 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:30.492928 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:30.492928 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:30.492928 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:30.492928 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:30.492928 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:30.492928 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:13:30.493514 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 86/163] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:30.502804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccbc3ae-32fb-46ac-9185-0944af140acf None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:30.517692 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be1dbe-d12b-4dbf-89c9-20b7087f4788 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:30.519195 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be1dbe-d12b-4dbf-89c9-20b7087f4788 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:30.530055 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:30.682649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccbc3ae-32fb-46ac-9185-0944af140acf None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:30.707841 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 78/164] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:13:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:13:30.709609 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be1dbe-d12b-4dbf-89c9-20b7087f4788 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:30.727075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:30.743063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc905952-4f75-430d-bd03-4bfb860c485e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:30.743708 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc905952-4f75-430d-bd03-4bfb860c485e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:30.744038 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc905952-4f75-430d-bd03-4bfb860c485e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:30.744356 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc905952-4f75-430d-bd03-4bfb860c485e None admin] PATH_INFO: `/v3/users/9da1572dde3847a2b938a4c8ed028505` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:30.747147 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc905952-4f75-430d-bd03-4bfb860c485e None admin] RBAC: Authorizing `identity:get_user(user_id=9da1572dde3847a2b938a4c8ed028505)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:30.748607 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc905952-4f75-430d-bd03-4bfb860c485e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:30.750224 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 87/165] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:30 2022] GET /identity/v3/users/9da1572dde3847a2b938a4c8ed028505 => generated 289 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:30.756387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc905952-4f75-430d-bd03-4bfb860c485e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:30.895947 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be1dbe-d12b-4dbf-89c9-20b7087f4788 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:30.905725 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:30.942290 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc905952-4f75-430d-bd03-4bfb860c485e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:30.960196 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fba8a5d-b6af-4326-9949-0eac6d3a7a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:30.960803 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba8a5d-b6af-4326-9949-0eac6d3a7a6d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:30.961077 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba8a5d-b6af-4326-9949-0eac6d3a7a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:30.961391 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fba8a5d-b6af-4326-9949-0eac6d3a7a6d None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:30.962056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fba8a5d-b6af-4326-9949-0eac6d3a7a6d None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:30.962351 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fba8a5d-b6af-4326-9949-0eac6d3a7a6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:30.963962 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fba8a5d-b6af-4326-9949-0eac6d3a7a6d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:30.978663 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 79/166] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Sep 2 09:13:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:30.980892 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fba8a5d-b6af-4326-9949-0eac6d3a7a6d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:31.008431 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca80008f-d2aa-49a1-ac20-a4cd5be192d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:31.009125 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca80008f-d2aa-49a1-ac20-a4cd5be192d1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:31.009402 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca80008f-d2aa-49a1-ac20-a4cd5be192d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:31.009671 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca80008f-d2aa-49a1-ac20-a4cd5be192d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:31.030427 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca80008f-d2aa-49a1-ac20-a4cd5be192d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:31.036574 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ca80008f-d2aa-49a1-ac20-a4cd5be192d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:31.036574 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:31.036574 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:31.036574 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:31.036574 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:31.036574 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:31.036574 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:31.036574 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:31.036574 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:31.036574 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:31.036574 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:31.036574 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:31.036574 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:31.036574 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:31.036574 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:31.036574 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:31.037448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:31.037448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:31.037448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:31.037448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:31.037448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:31.037448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:31.037448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:31.037448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:31.037448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:31.037448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:31.037448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:31.037448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:31.037448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:31.037448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:31.037448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:31.037448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:31.038326 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:31.038326 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:31.038326 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:31.038326 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:31.038326 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:31.038326 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:31.038326 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:31.038326 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:31.038326 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:31.038326 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:31.038326 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:31.038326 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:13:31.041552 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 88/167] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:31.042874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca80008f-d2aa-49a1-ac20-a4cd5be192d1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:31.098920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:31.170672 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fba8a5d-b6af-4326-9949-0eac6d3a7a6d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:31.220486 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca80008f-d2aa-49a1-ac20-a4cd5be192d1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:31.233070 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f39ab9-6f75-47b8-92cb-ba468e5a7195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:31.233713 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f39ab9-6f75-47b8-92cb-ba468e5a7195 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:31.233990 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f39ab9-6f75-47b8-92cb-ba468e5a7195 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:31.234261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f39ab9-6f75-47b8-92cb-ba468e5a7195 None admin] PATH_INFO: `/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:31.235428 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f39ab9-6f75-47b8-92cb-ba468e5a7195 None admin] RBAC: Authorizing `identity:get_project(project_id=79a2aa8010c54ad7ad7b27f08f4de74c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:31.236866 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f39ab9-6f75-47b8-92cb-ba468e5a7195 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:31.240734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 80/168] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:30 2022] GET /identity/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:31.241337 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f39ab9-6f75-47b8-92cb-ba468e5a7195 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:31.285219 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2236714-c837-4c76-ad49-031aeb56d22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:31.285863 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2236714-c837-4c76-ad49-031aeb56d22c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:31.286141 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2236714-c837-4c76-ad49-031aeb56d22c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:31.286412 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2236714-c837-4c76-ad49-031aeb56d22c None admin] PATH_INFO: `/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:31.287833 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2236714-c837-4c76-ad49-031aeb56d22c None admin] RBAC: Authorizing `identity:get_project(project_id=79a2aa8010c54ad7ad7b27f08f4de74c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:31.289276 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2236714-c837-4c76-ad49-031aeb56d22c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:31.298192 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 89/169] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:31 2022] GET /identity/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:31.299745 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2236714-c837-4c76-ad49-031aeb56d22c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:31.315070 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:31.396892 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f39ab9-6f75-47b8-92cb-ba468e5a7195 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:31.457834 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:31.467043 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcbcd92-1d57-4744-9b76-ee98fcc94ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:31.467707 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcbcd92-1d57-4744-9b76-ee98fcc94ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:31.467984 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcbcd92-1d57-4744-9b76-ee98fcc94ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:31.468255 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcbcd92-1d57-4744-9b76-ee98fcc94ec1 None admin] PATH_INFO: `/v3/users/604723d5b48f4e869bccbe56651aa80b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:31.469461 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcbcd92-1d57-4744-9b76-ee98fcc94ec1 None admin] RBAC: Authorizing `identity:get_user(user_id=604723d5b48f4e869bccbe56651aa80b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:31.476011 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcbcd92-1d57-4744-9b76-ee98fcc94ec1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:31.476011 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 81/170] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:31 2022] GET /identity/v3/users/604723d5b48f4e869bccbe56651aa80b => generated 275 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:31.476011 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcbcd92-1d57-4744-9b76-ee98fcc94ec1 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:31.505430 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2236714-c837-4c76-ad49-031aeb56d22c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:31.579640 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7cecb-e6b5-48b1-9aba-790672aa7769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:31.580280 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7cecb-e6b5-48b1-9aba-790672aa7769 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:31.580564 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7cecb-e6b5-48b1-9aba-790672aa7769 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:31.580844 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7cecb-e6b5-48b1-9aba-790672aa7769 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:31.581571 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7cecb-e6b5-48b1-9aba-790672aa7769 None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:31.581881 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7cecb-e6b5-48b1-9aba-790672aa7769 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:31.586995 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d7cecb-e6b5-48b1-9aba-790672aa7769 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:31.600947 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 90/171] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Sep 2 09:13:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:31.601961 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7cecb-e6b5-48b1-9aba-790672aa7769 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:31.620257 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcbcd92-1d57-4744-9b76-ee98fcc94ec1 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:31.686831 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8d1a2-c4f0-4384-8336-51ea07a41145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:31.687450 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8d1a2-c4f0-4384-8336-51ea07a41145 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:31.687727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8d1a2-c4f0-4384-8336-51ea07a41145 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:31.687995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8d1a2-c4f0-4384-8336-51ea07a41145 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:31.688733 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8d1a2-c4f0-4384-8336-51ea07a41145 None admin] RBAC: Adding query filter params (role.id=f2fe9fb31024471e9e2bea2b12715de6, scope.project.id=79a2aa8010c54ad7ad7b27f08f4de74c, user.id=604723d5b48f4e869bccbe56651aa80b) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:31.689039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8d1a2-c4f0-4384-8336-51ea07a41145 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:31.690479 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8d1a2-c4f0-4384-8336-51ea07a41145 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:31.699654 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 82/172] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:13:31 2022] GET /identity/v3/role_assignments?user.id=604723d5b48f4e869bccbe56651aa80b&scope.project.id=79a2aa8010c54ad7ad7b27f08f4de74c&role.id=f2fe9fb31024471e9e2bea2b12715de6 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:31.702449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8d1a2-c4f0-4384-8336-51ea07a41145 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:31.715184 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:31.779600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7cecb-e6b5-48b1-9aba-790672aa7769 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:31.827328 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:31.836508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a090ee-ec9b-4e9e-8500-7bede5183eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:31.837587 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a090ee-ec9b-4e9e-8500-7bede5183eb7 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:31.838009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a090ee-ec9b-4e9e-8500-7bede5183eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:31.838412 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a090ee-ec9b-4e9e-8500-7bede5183eb7 None admin] PATH_INFO: `/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c/users/9da1572dde3847a2b938a4c8ed028505/roles/b381d11646e54d1197676600202a6144` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:31.843758 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a090ee-ec9b-4e9e-8500-7bede5183eb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=79a2aa8010c54ad7ad7b27f08f4de74c, user_id=9da1572dde3847a2b938a4c8ed028505, role_id=b381d11646e54d1197676600202a6144)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:31.850734 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a090ee-ec9b-4e9e-8500-7bede5183eb7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:31.884740 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8d1a2-c4f0-4384-8336-51ea07a41145 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:31.930869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 91/173] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Sep 2 09:13:31 2022] PUT /identity/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c/users/9da1572dde3847a2b938a4c8ed028505/roles/b381d11646e54d1197676600202a6144 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:13:31.931244 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb33573-6034-4ab4-82d1-3a8f450a9dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:31.931949 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a090ee-ec9b-4e9e-8500-7bede5183eb7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:31.932057 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb33573-6034-4ab4-82d1-3a8f450a9dbe None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:31.932248 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb33573-6034-4ab4-82d1-3a8f450a9dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:31.932526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb33573-6034-4ab4-82d1-3a8f450a9dbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:32.075092 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb33573-6034-4ab4-82d1-3a8f450a9dbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:32.076532 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb33573-6034-4ab4-82d1-3a8f450a9dbe None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:32.133367 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a090ee-ec9b-4e9e-8500-7bede5183eb7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:32.170070 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:32.173832 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:32.182795 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2360ed22-ba50-4279-a3d4-83da15d5a165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:32.192983 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360ed22-ba50-4279-a3d4-83da15d5a165 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:32.192983 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360ed22-ba50-4279-a3d4-83da15d5a165 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:32.192983 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2360ed22-ba50-4279-a3d4-83da15d5a165 None admin] PATH_INFO: `/v3/users/604723d5b48f4e869bccbe56651aa80b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:32.192983 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360ed22-ba50-4279-a3d4-83da15d5a165 None admin] RBAC: Authorizing `identity:get_user(user_id=604723d5b48f4e869bccbe56651aa80b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:32.192983 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2360ed22-ba50-4279-a3d4-83da15d5a165 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:32.192983 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 92/174] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:31 2022] GET /identity/v3/users/604723d5b48f4e869bccbe56651aa80b => generated 275 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:32.193548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7adedf-4e9c-461e-98ab-0e8717a0a0e0 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:32.193548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7adedf-4e9c-461e-98ab-0e8717a0a0e0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:32.193548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7adedf-4e9c-461e-98ab-0e8717a0a0e0 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:32.193548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 93/175] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:32.196551 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7adedf-4e9c-461e-98ab-0e8717a0a0e0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:32.234502 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 83/176] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:13:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:13:32.235697 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6d47f-2f7a-48b2-bf4c-87ba5daf0fe3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:32.235908 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6d47f-2f7a-48b2-bf4c-87ba5daf0fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:32.236186 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c6d47f-2f7a-48b2-bf4c-87ba5daf0fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:32.270668 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a0c6d47f-2f7a-48b2-bf4c-87ba5daf0fe3 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:32.271905 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a0c6d47f-2f7a-48b2-bf4c-87ba5daf0fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:32.379921 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7adedf-4e9c-461e-98ab-0e8717a0a0e0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:32.404584 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 84/177] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:32.405619 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6d47f-2f7a-48b2-bf4c-87ba5daf0fe3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:32.444739 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387627e3-b0b8-4358-9c9c-64cea8d12ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:32.449190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387627e3-b0b8-4358-9c9c-64cea8d12ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:32.449472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387627e3-b0b8-4358-9c9c-64cea8d12ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:32.449746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387627e3-b0b8-4358-9c9c-64cea8d12ce6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:32.540172 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:32.547575 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c6d47f-2f7a-48b2-bf4c-87ba5daf0fe3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:32.561489 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:32.567812 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387627e3-b0b8-4358-9c9c-64cea8d12ce6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:32.569622 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387627e3-b0b8-4358-9c9c-64cea8d12ce6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:32.592392 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e8242a-868f-46e2-bcaf-d48462e91ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:32.593026 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8242a-868f-46e2-bcaf-d48462e91ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:32.599940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8242a-868f-46e2-bcaf-d48462e91ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:32.599940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e8242a-868f-46e2-bcaf-d48462e91ff0 None admin] PATH_INFO: `/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:32.599940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e8242a-868f-46e2-bcaf-d48462e91ff0 None admin] RBAC: Authorizing `identity:get_project(project_id=79a2aa8010c54ad7ad7b27f08f4de74c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:32.599940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e8242a-868f-46e2-bcaf-d48462e91ff0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:32.599940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 85/178] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:32 2022] GET /identity/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:32.603966 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed121c3c-8647-4f35-aae0-2c11262d06f7 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:32.604253 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed121c3c-8647-4f35-aae0-2c11262d06f7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:32.604555 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed121c3c-8647-4f35-aae0-2c11262d06f7 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:32.605862 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 86/179] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:32.606908 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049b861-50c3-4eff-b7d9-08e244a2ca00 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:32.607192 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049b861-50c3-4eff-b7d9-08e244a2ca00 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:32.607466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7049b861-50c3-4eff-b7d9-08e244a2ca00 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:32.608825 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 87/180] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:32.611548 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049b861-50c3-4eff-b7d9-08e244a2ca00 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:32.689645 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 94/181] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:13:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:13:32.690903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16778ae-6ac5-416d-bc57-327ca946e6e5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:32.691115 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16778ae-6ac5-416d-bc57-327ca946e6e5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:32.691371 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16778ae-6ac5-416d-bc57-327ca946e6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:32.710865 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f16778ae-6ac5-416d-bc57-327ca946e6e5 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:32.712418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f16778ae-6ac5-416d-bc57-327ca946e6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:32.735677 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7049b861-50c3-4eff-b7d9-08e244a2ca00 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:32.771319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa41709-6331-4ab9-b3c3-70d943bee7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:32.771929 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa41709-6331-4ab9-b3c3-70d943bee7ba None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:32.772207 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa41709-6331-4ab9-b3c3-70d943bee7ba None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:32.772477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa41709-6331-4ab9-b3c3-70d943bee7ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:32.773204 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa41709-6331-4ab9-b3c3-70d943bee7ba None admin] RBAC: Adding query filter params (role.id=efe6bdd5112340b48b262ceafbc19c04, scope.project.id=79a2aa8010c54ad7ad7b27f08f4de74c, user.id=604723d5b48f4e869bccbe56651aa80b) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:32.773507 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa41709-6331-4ab9-b3c3-70d943bee7ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:32.774969 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa41709-6331-4ab9-b3c3-70d943bee7ba None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:32.780915 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 88/182] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:13:32 2022] GET /identity/v3/role_assignments?user.id=604723d5b48f4e869bccbe56651aa80b&scope.project.id=79a2aa8010c54ad7ad7b27f08f4de74c&role.id=efe6bdd5112340b48b262ceafbc19c04 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:32.782058 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7371bd-6e0c-49da-a681-4b246a12fa50 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:32.782335 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7371bd-6e0c-49da-a681-4b246a12fa50 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:32.782602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7371bd-6e0c-49da-a681-4b246a12fa50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:32.803418 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ba7371bd-6e0c-49da-a681-4b246a12fa50 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:32.804677 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ba7371bd-6e0c-49da-a681-4b246a12fa50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:32.808172 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 95/183] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:32.810369 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16778ae-6ac5-416d-bc57-327ca946e6e5 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:32.829875 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:32.883054 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:32.896914 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 89/184] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:32.898098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7371bd-6e0c-49da-a681-4b246a12fa50 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:33.014662 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7371bd-6e0c-49da-a681-4b246a12fa50 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:33.024398 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16778ae-6ac5-416d-bc57-327ca946e6e5 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:33.055187 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fff53d-2d24-41b6-aabe-006b1e16eb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:33.056189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fff53d-2d24-41b6-aabe-006b1e16eb91 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:33.056917 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fff53d-2d24-41b6-aabe-006b1e16eb91 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:33.057393 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fff53d-2d24-41b6-aabe-006b1e16eb91 None admin] PATH_INFO: `/v3/projects/883aa06e33b5424b90208ef34b521e0d` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:33.059437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fff53d-2d24-41b6-aabe-006b1e16eb91 None admin] RBAC: Authorizing `identity:get_project(project_id=883aa06e33b5424b90208ef34b521e0d)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:33.061079 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fff53d-2d24-41b6-aabe-006b1e16eb91 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:33.063021 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 90/185] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:32 2022] GET /identity/v3/projects/883aa06e33b5424b90208ef34b521e0d => generated 312 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:33.069130 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fff53d-2d24-41b6-aabe-006b1e16eb91 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:33.106099 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:33.107839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6bd2f5-2e81-4bc1-9c27-1b2eecad2a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:33.108490 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bd2f5-2e81-4bc1-9c27-1b2eecad2a8f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:33.108766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bd2f5-2e81-4bc1-9c27-1b2eecad2a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:33.109033 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6bd2f5-2e81-4bc1-9c27-1b2eecad2a8f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:33.132716 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6bd2f5-2e81-4bc1-9c27-1b2eecad2a8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:33.137632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-fe6bd2f5-2e81-4bc1-9c27-1b2eecad2a8f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:13:33.137632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:33.137632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:33.137632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:33.137632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:33.137632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:33.137632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:33.137632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:33.137632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:33.137632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:33.137632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:33.137632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:33.137632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:33.137632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:33.137632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:33.137632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:33.138512 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:33.138512 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:33.138512 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:33.138512 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:33.138512 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:33.138512 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:33.138512 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:33.138512 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:33.138512 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:33.138512 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:33.138512 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:33.138512 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:33.138512 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:33.138512 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:33.138512 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:33.138512 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:33.139336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:33.139336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:33.139336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:33.139336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:33.139336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:33.139336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:33.139336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:33.139336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:33.139336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:33.139336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:33.139336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:13:33.139336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:13:33.143071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 96/186] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:13:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:33.148731 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6bd2f5-2e81-4bc1-9c27-1b2eecad2a8f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:33.188494 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fff53d-2d24-41b6-aabe-006b1e16eb91 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:33.225165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad5f6c9-340f-4baf-b2d5-f722da17284b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:33.226159 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5f6c9-340f-4baf-b2d5-f722da17284b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:33.226641 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5f6c9-340f-4baf-b2d5-f722da17284b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:33.227135 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad5f6c9-340f-4baf-b2d5-f722da17284b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:33.228219 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad5f6c9-340f-4baf-b2d5-f722da17284b None admin] RBAC: Adding query filter params (role.id=efe6bdd5112340b48b262ceafbc19c04, scope.project.id=883aa06e33b5424b90208ef34b521e0d, user.id=604723d5b48f4e869bccbe56651aa80b) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:33.228889 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad5f6c9-340f-4baf-b2d5-f722da17284b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:33.231172 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad5f6c9-340f-4baf-b2d5-f722da17284b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:33.238344 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 91/187] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:13:33 2022] GET /identity/v3/role_assignments?user.id=604723d5b48f4e869bccbe56651aa80b&scope.project.id=883aa06e33b5424b90208ef34b521e0d&role.id=efe6bdd5112340b48b262ceafbc19c04 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:33.292242 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad5f6c9-340f-4baf-b2d5-f722da17284b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:33.335353 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:33.361871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6bd2f5-2e81-4bc1-9c27-1b2eecad2a8f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:33.411387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc533f99-f62b-42aa-a84c-2d9ca92c98ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:33.412059 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc533f99-f62b-42aa-a84c-2d9ca92c98ed None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:33.412336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc533f99-f62b-42aa-a84c-2d9ca92c98ed None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:33.412609 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc533f99-f62b-42aa-a84c-2d9ca92c98ed None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:33.413316 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc533f99-f62b-42aa-a84c-2d9ca92c98ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:33.413617 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc533f99-f62b-42aa-a84c-2d9ca92c98ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:33.415119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc533f99-f62b-42aa-a84c-2d9ca92c98ed None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:33.420890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:33.424723 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 97/188] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:33.426194 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc533f99-f62b-42aa-a84c-2d9ca92c98ed None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:33.439488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad5f6c9-340f-4baf-b2d5-f722da17284b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:33.483882 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8493b-08cb-4b8b-b950-3aeffbb1594c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:33.484641 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8493b-08cb-4b8b-b950-3aeffbb1594c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:33.484922 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8493b-08cb-4b8b-b950-3aeffbb1594c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:33.485196 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a8493b-08cb-4b8b-b950-3aeffbb1594c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:33.486391 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a8493b-08cb-4b8b-b950-3aeffbb1594c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:33.488959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a8493b-08cb-4b8b-b950-3aeffbb1594c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:33.491317 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 92/189] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:13:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:33.491317 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8493b-08cb-4b8b-b950-3aeffbb1594c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:33.556589 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc533f99-f62b-42aa-a84c-2d9ca92c98ed None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:33.594508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08350e-b3a2-4d15-aa2d-55d52c4eb891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:33.595844 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08350e-b3a2-4d15-aa2d-55d52c4eb891 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:33.596123 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08350e-b3a2-4d15-aa2d-55d52c4eb891 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:33.596397 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08350e-b3a2-4d15-aa2d-55d52c4eb891 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:33.607594 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08350e-b3a2-4d15-aa2d-55d52c4eb891 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:33.611496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a8493b-08cb-4b8b-b950-3aeffbb1594c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:33.613571 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ff08350e-b3a2-4d15-aa2d-55d52c4eb891 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 09:13:33.613571 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:33.613571 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:33.613571 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:33.613571 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:33.613571 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:33.613571 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:33.613571 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:33.613571 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:33.613571 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:33.613571 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:33.613571 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:33.613571 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:33.613571 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:33.613571 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:33.613571 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:33.614433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:33.614433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:33.614433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:33.614433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:33.614433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:33.614433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:33.614433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:33.614433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:33.614433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:33.614433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:33.614433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:33.614433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:33.614433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:33.614433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:33.614433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:33.614433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:33.615248 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:33.615248 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:33.615248 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:33.615248 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:33.615248 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:33.615248 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:33.615248 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:33.615248 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:33.615248 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:33.615248 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:33.615248 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 09:13:33.615248 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:13:33.617140 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 98/190] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Sep 2 09:13:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:33.618031 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08350e-b3a2-4d15-aa2d-55d52c4eb891 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:33.632071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:33.650172 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc47ce-fb58-4770-925b-455be72dfe35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:33.650835 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc47ce-fb58-4770-925b-455be72dfe35 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:33.651171 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc47ce-fb58-4770-925b-455be72dfe35 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:33.651476 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc47ce-fb58-4770-925b-455be72dfe35 None admin] PATH_INFO: `/v3/users/941ab0618914451992a54b0b138a7a70` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:33.667630 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bc47ce-fb58-4770-925b-455be72dfe35 None admin] RBAC: Authorizing `identity:get_user(user_id=941ab0618914451992a54b0b138a7a70)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:33.669462 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bc47ce-fb58-4770-925b-455be72dfe35 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:33.671542 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 93/191] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:33 2022] GET /identity/v3/users/941ab0618914451992a54b0b138a7a70 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:33.676477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc47ce-fb58-4770-925b-455be72dfe35 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:33.690098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:33.735763 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08350e-b3a2-4d15-aa2d-55d52c4eb891 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:33.770285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa688a-ce13-4cbb-8f31-803c3977447e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:33.770930 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa688a-ce13-4cbb-8f31-803c3977447e None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:33.771208 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa688a-ce13-4cbb-8f31-803c3977447e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:33.771482 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa688a-ce13-4cbb-8f31-803c3977447e None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:33.772131 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefa688a-ce13-4cbb-8f31-803c3977447e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:33.773586 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefa688a-ce13-4cbb-8f31-803c3977447e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:33.777356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc47ce-fb58-4770-925b-455be72dfe35 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:33.814937 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e5842-af1f-43ca-ae22-a20a1ddb88c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:33.814937 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e5842-af1f-43ca-ae22-a20a1ddb88c2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:33.814937 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e5842-af1f-43ca-ae22-a20a1ddb88c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:33.814937 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e5842-af1f-43ca-ae22-a20a1ddb88c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:33.815421 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721e5842-af1f-43ca-ae22-a20a1ddb88c2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:33.815421 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721e5842-af1f-43ca-ae22-a20a1ddb88c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:33.815680 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721e5842-af1f-43ca-ae22-a20a1ddb88c2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:33.823943 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 94/192] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:13:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:33.824598 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e5842-af1f-43ca-ae22-a20a1ddb88c2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:33.901649 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:33.923775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e5842-af1f-43ca-ae22-a20a1ddb88c2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:33.956299 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3c7b8-db66-43c8-9333-52a469c2c011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:33.956905 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3c7b8-db66-43c8-9333-52a469c2c011 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:33.957181 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3c7b8-db66-43c8-9333-52a469c2c011 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:33.957452 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3c7b8-db66-43c8-9333-52a469c2c011 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:34.021898 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3c7b8-db66-43c8-9333-52a469c2c011 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:34.023389 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a3c7b8-db66-43c8-9333-52a469c2c011 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:34.116240 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 95/193] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:13:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:13:34.117073 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3c7b8-db66-43c8-9333-52a469c2c011 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:34.135774 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 99/194] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Sep 2 09:13:33 2022] POST /identity/v3/users => generated 297 bytes in 518 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:13:34.148321 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692f577-bb05-4dde-adc1-4d8c6989951c None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:34.148607 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692f577-bb05-4dde-adc1-4d8c6989951c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:34.148881 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692f577-bb05-4dde-adc1-4d8c6989951c None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:34.157323 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 100/195] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:34.157323 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692f577-bb05-4dde-adc1-4d8c6989951c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:34.158506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:34.226585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a3c7b8-db66-43c8-9333-52a469c2c011 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:34.227504 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:34.267380 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010508f0-c23a-40fc-8bab-015e55a8de58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:34.268026 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010508f0-c23a-40fc-8bab-015e55a8de58 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:34.268303 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010508f0-c23a-40fc-8bab-015e55a8de58 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:34.268563 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010508f0-c23a-40fc-8bab-015e55a8de58 None admin] PATH_INFO: `/v3/users/941ab0618914451992a54b0b138a7a70` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:34.269691 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010508f0-c23a-40fc-8bab-015e55a8de58 None admin] RBAC: Authorizing `identity:get_user(user_id=941ab0618914451992a54b0b138a7a70)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:34.270026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692f577-bb05-4dde-adc1-4d8c6989951c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:34.271243 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010508f0-c23a-40fc-8bab-015e55a8de58 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:34.272855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 96/196] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:34 2022] GET /identity/v3/users/941ab0618914451992a54b0b138a7a70 => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:34.272855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f42fc-53bf-4c18-9937-6afd673c15ea None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:34.272855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f42fc-53bf-4c18-9937-6afd673c15ea None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:34.272855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f42fc-53bf-4c18-9937-6afd673c15ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:34.295117 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1d1f42fc-53bf-4c18-9937-6afd673c15ea None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:34.296315 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1d1f42fc-53bf-4c18-9937-6afd673c15ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:34.314500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a45bc9-2a69-4afa-bf12-7ad21a6f9724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:34.315198 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a45bc9-2a69-4afa-bf12-7ad21a6f9724 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:34.315848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a45bc9-2a69-4afa-bf12-7ad21a6f9724 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:34.315848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a45bc9-2a69-4afa-bf12-7ad21a6f9724 None admin] PATH_INFO: `/v3/projects/dac3a1a96e914a549133e73f03943618` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:34.317444 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a45bc9-2a69-4afa-bf12-7ad21a6f9724 None admin] RBAC: Authorizing `identity:get_project(project_id=dac3a1a96e914a549133e73f03943618)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:34.318771 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a45bc9-2a69-4afa-bf12-7ad21a6f9724 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:34.320885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 101/197] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:34 2022] GET /identity/v3/projects/dac3a1a96e914a549133e73f03943618 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:34.321657 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a45bc9-2a69-4afa-bf12-7ad21a6f9724 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:34.348712 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:34.375921 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 97/198] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:34.376783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f42fc-53bf-4c18-9937-6afd673c15ea None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:34.448892 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a45bc9-2a69-4afa-bf12-7ad21a6f9724 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:34.483815 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:34.492150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b62b3-7377-446c-8ae7-bcec7e37a573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:34.492770 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b62b3-7377-446c-8ae7-bcec7e37a573 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:34.496712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b62b3-7377-446c-8ae7-bcec7e37a573 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:34.496712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437b62b3-7377-446c-8ae7-bcec7e37a573 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:34.506602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f42fc-53bf-4c18-9937-6afd673c15ea None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:34.548380 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00efce99-dcc1-4599-a83a-60754dc80d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:34.549274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efce99-dcc1-4599-a83a-60754dc80d08 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:34.549559 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efce99-dcc1-4599-a83a-60754dc80d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:34.549826 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efce99-dcc1-4599-a83a-60754dc80d08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:34.550760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00efce99-dcc1-4599-a83a-60754dc80d08 None admin] RBAC: Adding query filter params (role.id=f886b8e8889549e896a8fb3426cce9e1, scope.project.id=dac3a1a96e914a549133e73f03943618, user.id=941ab0618914451992a54b0b138a7a70) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:34.551079 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00efce99-dcc1-4599-a83a-60754dc80d08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:34.552825 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00efce99-dcc1-4599-a83a-60754dc80d08 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:34.562247 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 98/199] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:13:34 2022] GET /identity/v3/role_assignments?user.id=941ab0618914451992a54b0b138a7a70&scope.project.id=dac3a1a96e914a549133e73f03943618&role.id=f886b8e8889549e896a8fb3426cce9e1 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:34.572567 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b62b3-7377-446c-8ae7-bcec7e37a573 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:34.574620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437b62b3-7377-446c-8ae7-bcec7e37a573 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:34.690695 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 102/200] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:13:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:13:34.698794 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00efce99-dcc1-4599-a83a-60754dc80d08 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:34.726687 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b62b3-7377-446c-8ae7-bcec7e37a573 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:34.760284 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:34.767075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:34.886618 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00efce99-dcc1-4599-a83a-60754dc80d08 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:34.891968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437b62b3-7377-446c-8ae7-bcec7e37a573 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:34.948725 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ef63d-30d5-401e-82f9-c2adb6d840c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:34.949327 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ef63d-30d5-401e-82f9-c2adb6d840c6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:34.949554 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ef63d-30d5-401e-82f9-c2adb6d840c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:34.949825 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ef63d-30d5-401e-82f9-c2adb6d840c6 None admin] PATH_INFO: `/v3/projects/dac3a1a96e914a549133e73f03943618` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:34.952097 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f646cf55-5264-49c5-a7fb-8ba11b7ceab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:34.952730 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646cf55-5264-49c5-a7fb-8ba11b7ceab7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:34.953002 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646cf55-5264-49c5-a7fb-8ba11b7ceab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:34.953268 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646cf55-5264-49c5-a7fb-8ba11b7ceab7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:34.954375 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ef63d-30d5-401e-82f9-c2adb6d840c6 None admin] RBAC: Authorizing `identity:get_project(project_id=dac3a1a96e914a549133e73f03943618)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:34.955846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ef63d-30d5-401e-82f9-c2adb6d840c6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:34.960049 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 99/201] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:34 2022] GET /identity/v3/projects/dac3a1a96e914a549133e73f03943618 => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:34.964702 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ef63d-30d5-401e-82f9-c2adb6d840c6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:34.979309 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f646cf55-5264-49c5-a7fb-8ba11b7ceab7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:34.984685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-f646cf55-5264-49c5-a7fb-8ba11b7ceab7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 09:13:34.984685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:34.984685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:34.984685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:34.984685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:34.984685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:34.984685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:34.984685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:34.984685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:34.984685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:34.984685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:34.984685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:34.984685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:34.984685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:34.984685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:34.984685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:34.985605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:34.985605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:34.985605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:34.985605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:34.985605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:34.985605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:34.985605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:34.985605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:34.985605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:34.985605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:34.985605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:34.985605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:34.985605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:34.985605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:34.985605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:34.985605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:34.986418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:34.986418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:34.986418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:34.986418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:34.986418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:34.986418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:34.986418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:34.986418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:34.986418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:34.986418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:34.986418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 09:13:34.986418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:13:34.986418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 103/202] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:13:34.996294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f646cf55-5264-49c5-a7fb-8ba11b7ceab7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:35.152828 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:35.159725 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ef63d-30d5-401e-82f9-c2adb6d840c6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:35.170085 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:35.173413 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f646cf55-5264-49c5-a7fb-8ba11b7ceab7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:35.218194 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5f842-7864-49ca-8c22-3fa9b0654fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:35.218755 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5f842-7864-49ca-8c22-3fa9b0654fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:35.219064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5f842-7864-49ca-8c22-3fa9b0654fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:35.219336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5f842-7864-49ca-8c22-3fa9b0654fb0 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:35.220009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d5f842-7864-49ca-8c22-3fa9b0654fb0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:35.220309 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d5f842-7864-49ca-8c22-3fa9b0654fb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:35.221691 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d5f842-7864-49ca-8c22-3fa9b0654fb0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:35.231780 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 104/203] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Sep 2 09:13:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:35.233032 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e86969-83ed-4de4-a3ba-f0880b475de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:35.233652 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e86969-83ed-4de4-a3ba-f0880b475de5 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:35.233927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e86969-83ed-4de4-a3ba-f0880b475de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:35.234195 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e86969-83ed-4de4-a3ba-f0880b475de5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:35.242748 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e86969-83ed-4de4-a3ba-f0880b475de5 None admin] RBAC: Adding query filter params (role.id=f2fe9fb31024471e9e2bea2b12715de6, scope.project.id=dac3a1a96e914a549133e73f03943618, user.id=941ab0618914451992a54b0b138a7a70) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:35.242748 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e86969-83ed-4de4-a3ba-f0880b475de5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:35.242748 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e86969-83ed-4de4-a3ba-f0880b475de5 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:35.243156 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 100/204] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:13:34 2022] GET /identity/v3/role_assignments?user.id=941ab0618914451992a54b0b138a7a70&scope.project.id=dac3a1a96e914a549133e73f03943618&role.id=f2fe9fb31024471e9e2bea2b12715de6 => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:35.243735 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5f842-7864-49ca-8c22-3fa9b0654fb0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:35.411118 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5f842-7864-49ca-8c22-3fa9b0654fb0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:35.446267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dadd0a-c2d2-4bce-8f37-66d6d177a393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:35.446965 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dadd0a-c2d2-4bce-8f37-66d6d177a393 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:35.447293 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dadd0a-c2d2-4bce-8f37-66d6d177a393 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:35.447571 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dadd0a-c2d2-4bce-8f37-66d6d177a393 None admin] PATH_INFO: `/v3/users/9da1572dde3847a2b938a4c8ed028505` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:35.449236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dadd0a-c2d2-4bce-8f37-66d6d177a393 None admin] RBAC: Authorizing `identity:get_user(user_id=9da1572dde3847a2b938a4c8ed028505)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:35.450676 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dadd0a-c2d2-4bce-8f37-66d6d177a393 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:35.452706 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 105/205] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:35 2022] GET /identity/v3/users/9da1572dde3847a2b938a4c8ed028505 => generated 289 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:35.458332 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e86969-83ed-4de4-a3ba-f0880b475de5 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:35.611814 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e86969-83ed-4de4-a3ba-f0880b475de5 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:35.659876 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b92624-34ce-48a3-9290-50f8c20d9d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:35.660495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b92624-34ce-48a3-9290-50f8c20d9d66 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:35.660774 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b92624-34ce-48a3-9290-50f8c20d9d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:35.661041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b92624-34ce-48a3-9290-50f8c20d9d66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:35.719226 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:35.754406 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b92624-34ce-48a3-9290-50f8c20d9d66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:35.755989 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b92624-34ce-48a3-9290-50f8c20d9d66 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:35.898448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 101/206] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:13:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:13:35.904525 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dadd0a-c2d2-4bce-8f37-66d6d177a393 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:35.937303 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:36.040856 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dadd0a-c2d2-4bce-8f37-66d6d177a393 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:36.084799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47f3fc-557d-4f3d-9594-baf5c709741c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:36.085441 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47f3fc-557d-4f3d-9594-baf5c709741c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:36.085721 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47f3fc-557d-4f3d-9594-baf5c709741c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:36.085993 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47f3fc-557d-4f3d-9594-baf5c709741c None admin] PATH_INFO: `/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:36.087163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe47f3fc-557d-4f3d-9594-baf5c709741c None admin] RBAC: Authorizing `identity:get_project(project_id=79a2aa8010c54ad7ad7b27f08f4de74c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:36.088589 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe47f3fc-557d-4f3d-9594-baf5c709741c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:36.095809 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 106/207] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:35 2022] GET /identity/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:36.101696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b92624-34ce-48a3-9290-50f8c20d9d66 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:36.183558 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:36.225628 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b92624-34ce-48a3-9290-50f8c20d9d66 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:36.271291 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e380bb4-25c5-41f1-a397-f0e3f5ef0477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:36.271903 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e380bb4-25c5-41f1-a397-f0e3f5ef0477 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:36.272184 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e380bb4-25c5-41f1-a397-f0e3f5ef0477 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:36.272457 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e380bb4-25c5-41f1-a397-f0e3f5ef0477 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:36.273187 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e380bb4-25c5-41f1-a397-f0e3f5ef0477 None admin] RBAC: Adding query filter params (role.id=b381d11646e54d1197676600202a6144, scope.project.id=79a2aa8010c54ad7ad7b27f08f4de74c, user.id=9da1572dde3847a2b938a4c8ed028505) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:36.273493 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e380bb4-25c5-41f1-a397-f0e3f5ef0477 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:36.274950 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e380bb4-25c5-41f1-a397-f0e3f5ef0477 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:36.290882 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 102/208] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:13:36 2022] GET /identity/v3/role_assignments?user.id=9da1572dde3847a2b938a4c8ed028505&scope.project.id=79a2aa8010c54ad7ad7b27f08f4de74c&role.id=b381d11646e54d1197676600202a6144 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:36.573390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e739ed7-b4a7-4cf4-9a37-1b0fdfcb2849 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:36.573723 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e739ed7-b4a7-4cf4-9a37-1b0fdfcb2849 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:36.573908 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e739ed7-b4a7-4cf4-9a37-1b0fdfcb2849 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:36.575654 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 107/209] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:36.581457 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba898e7-6c44-4b2e-958f-afcec30ce81b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:36.581742 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba898e7-6c44-4b2e-958f-afcec30ce81b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:36.582018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba898e7-6c44-4b2e-958f-afcec30ce81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:36.633160 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5ba898e7-6c44-4b2e-958f-afcec30ce81b None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:36.634366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5ba898e7-6c44-4b2e-958f-afcec30ce81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:36.716426 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 103/210] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:36.732534 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b89837-db3a-44b7-a824-2605b24e1dce None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:36.732885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b89837-db3a-44b7-a824-2605b24e1dce None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:36.733170 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b89837-db3a-44b7-a824-2605b24e1dce None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:36.734736 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 108/211] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:36.739772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a203d0-f7f8-43d8-9b93-b5cb0e695bce None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:36.740063 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a203d0-f7f8-43d8-9b93-b5cb0e695bce None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:36.740336 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a203d0-f7f8-43d8-9b93-b5cb0e695bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:36.759399 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f0a203d0-f7f8-43d8-9b93-b5cb0e695bce None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:36.760682 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f0a203d0-f7f8-43d8-9b93-b5cb0e695bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:36.825763 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:36.834038 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 104/212] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:37.001906 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b89837-db3a-44b7-a824-2605b24e1dce None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:37.003642 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e224e0-f5b4-4fda-9776-04a1392313c2 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:37.003933 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e224e0-f5b4-4fda-9776-04a1392313c2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:37.004211 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e224e0-f5b4-4fda-9776-04a1392313c2 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:37.005656 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 105/213] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:37.014332 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604a988-6332-46d8-bf2b-67bd9c645916 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:37.014615 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604a988-6332-46d8-bf2b-67bd9c645916 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:37.014960 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604a988-6332-46d8-bf2b-67bd9c645916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:37.044530 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4604a988-6332-46d8-bf2b-67bd9c645916 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:37.045778 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4604a988-6332-46d8-bf2b-67bd9c645916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:37.119459 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 106/214] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:37.119459 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604a988-6332-46d8-bf2b-67bd9c645916 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:37.136155 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:37.147221 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b89837-db3a-44b7-a824-2605b24e1dce None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:37.186231 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b41b3-83ba-4a77-bd1a-722a30965839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:37.187202 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b41b3-83ba-4a77-bd1a-722a30965839 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:37.187564 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b41b3-83ba-4a77-bd1a-722a30965839 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:37.187910 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b41b3-83ba-4a77-bd1a-722a30965839 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:37.201007 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b41b3-83ba-4a77-bd1a-722a30965839 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:37.206837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2a8b41b3-83ba-4a77-bd1a-722a30965839 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:37.206837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:37.206837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:37.206837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:37.206837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:37.206837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:37.206837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:37.206837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:37.206837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:37.206837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:37.206837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:37.206837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:37.206837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:37.206837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:37.206837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:37.206837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:37.207946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:37.207946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:37.207946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:37.207946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:37.207946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:37.207946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:37.207946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:37.207946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:37.207946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:37.207946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:37.207946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:37.207946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:37.207946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:37.207946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:37.207946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:37.207946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:37.208964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:37.208964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:37.208964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:37.208964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:37.208964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:37.208964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:37.208964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:37.208964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:37.208964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:37.208964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:37.208964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:37.208964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:13:37.214754 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 109/215] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:37.217293 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b41b3-83ba-4a77-bd1a-722a30965839 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:37.253346 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604a988-6332-46d8-bf2b-67bd9c645916 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:37.279512 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:37.291513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e95cf-698f-47a6-a70e-69a2bcf3a2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:37.293010 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e95cf-698f-47a6-a70e-69a2bcf3a2dd None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:37.293574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e95cf-698f-47a6-a70e-69a2bcf3a2dd None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:37.294077 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1e95cf-698f-47a6-a70e-69a2bcf3a2dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:37.305935 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1e95cf-698f-47a6-a70e-69a2bcf3a2dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:37.312053 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-4c1e95cf-698f-47a6-a70e-69a2bcf3a2dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:13:37.312053 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:37.312053 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:37.312053 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:37.312053 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:37.312053 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:37.312053 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:37.312053 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:37.312053 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:37.312053 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:37.312053 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:37.312053 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:37.312053 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:37.312053 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:37.312053 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:37.312053 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:37.312999 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:37.312999 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:37.312999 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:37.312999 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:37.312999 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:37.312999 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:37.312999 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:37.312999 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:37.312999 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:37.312999 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:37.312999 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:37.312999 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:37.312999 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:37.312999 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:37.312999 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:37.312999 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:37.313819 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:37.313819 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:37.313819 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:37.313819 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:37.313819 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:37.313819 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:37.313819 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:37.313819 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:37.313819 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:37.313819 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:37.313819 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:13:37.313819 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:13:37.316093 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 107/216] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:13:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:37.321830 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e95cf-698f-47a6-a70e-69a2bcf3a2dd None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:37.331844 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b41b3-83ba-4a77-bd1a-722a30965839 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:37.366630 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a9cf5-5579-4d02-b000-66192e352a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:37.367280 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a9cf5-5579-4d02-b000-66192e352a18 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:37.367564 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a9cf5-5579-4d02-b000-66192e352a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:37.367839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a9cf5-5579-4d02-b000-66192e352a18 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:37.368505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a9cf5-5579-4d02-b000-66192e352a18 None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:37.368809 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a9cf5-5579-4d02-b000-66192e352a18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:37.370792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a9cf5-5579-4d02-b000-66192e352a18 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:37.382742 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 110/217] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Sep 2 09:13:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:37.387559 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a9cf5-5579-4d02-b000-66192e352a18 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:37.407962 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:37.441145 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1e95cf-698f-47a6-a70e-69a2bcf3a2dd None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:37.485002 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa5b4c-da2e-46e6-86f3-d6428b244e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:37.485915 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa5b4c-da2e-46e6-86f3-d6428b244e26 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:37.486203 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa5b4c-da2e-46e6-86f3-d6428b244e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:37.486639 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa5b4c-da2e-46e6-86f3-d6428b244e26 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:37.487345 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa5b4c-da2e-46e6-86f3-d6428b244e26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:37.487714 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa5b4c-da2e-46e6-86f3-d6428b244e26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:37.489125 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa5b4c-da2e-46e6-86f3-d6428b244e26 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:37.498871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 108/218] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:37.499664 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa5b4c-da2e-46e6-86f3-d6428b244e26 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:37.514520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a9cf5-5579-4d02-b000-66192e352a18 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:37.548412 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b77fb-1f45-4573-add0-16894d69634a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:37.553555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b77fb-1f45-4573-add0-16894d69634a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:37.553555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b77fb-1f45-4573-add0-16894d69634a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:37.553555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b77fb-1f45-4573-add0-16894d69634a None admin] PATH_INFO: `/v3/users/b28f835d8ca443768cc83703b41ae12b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:37.553778 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:37.565665 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b77fb-1f45-4573-add0-16894d69634a None admin] RBAC: Authorizing `identity:get_user(user_id=b28f835d8ca443768cc83703b41ae12b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:37.567161 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b77fb-1f45-4573-add0-16894d69634a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:37.568894 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 111/219] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:37 2022] GET /identity/v3/users/b28f835d8ca443768cc83703b41ae12b => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:37.574143 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b77fb-1f45-4573-add0-16894d69634a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:37.611079 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa5b4c-da2e-46e6-86f3-d6428b244e26 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:37.640357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:37.646353 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef915103-1aec-4186-9d38-085d2aaac33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:37.647006 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef915103-1aec-4186-9d38-085d2aaac33c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:37.648088 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef915103-1aec-4186-9d38-085d2aaac33c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:37.648363 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef915103-1aec-4186-9d38-085d2aaac33c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:37.650269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef915103-1aec-4186-9d38-085d2aaac33c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:37.651863 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef915103-1aec-4186-9d38-085d2aaac33c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:37.653420 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 109/220] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:13:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:37.654181 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef915103-1aec-4186-9d38-085d2aaac33c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:37.672235 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b77fb-1f45-4573-add0-16894d69634a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:37.706097 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ff5dd8-6ef7-4748-852f-e5319a1e5cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:37.706756 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff5dd8-6ef7-4748-852f-e5319a1e5cbb None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:37.707039 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff5dd8-6ef7-4748-852f-e5319a1e5cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:37.707316 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff5dd8-6ef7-4748-852f-e5319a1e5cbb None admin] PATH_INFO: `/v3/users/941ab0618914451992a54b0b138a7a70` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:37.708438 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff5dd8-6ef7-4748-852f-e5319a1e5cbb None admin] RBAC: Authorizing `identity:get_user(user_id=941ab0618914451992a54b0b138a7a70)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:37.709873 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ff5dd8-6ef7-4748-852f-e5319a1e5cbb None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:37.711490 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 112/221] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:37 2022] GET /identity/v3/users/941ab0618914451992a54b0b138a7a70 => generated 283 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:37.712604 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9879b-abd8-487a-822c-b8c382dc9921 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:37.712885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9879b-abd8-487a-822c-b8c382dc9921 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:37.713183 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9879b-abd8-487a-822c-b8c382dc9921 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:37.714412 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 113/222] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:37.715137 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9879b-abd8-487a-822c-b8c382dc9921 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:37.758689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef915103-1aec-4186-9d38-085d2aaac33c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:37.771917 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:37.790746 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c8c50-5d33-4c40-a4ec-d599a0fef9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:37.791458 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c8c50-5d33-4c40-a4ec-d599a0fef9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:37.792261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c8c50-5d33-4c40-a4ec-d599a0fef9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:37.792534 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c8c50-5d33-4c40-a4ec-d599a0fef9d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:37.817451 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9879b-abd8-487a-822c-b8c382dc9921 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:37.850394 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58edee-45a9-4c11-8994-85c31a769de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:37.851032 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58edee-45a9-4c11-8994-85c31a769de0 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:37.851309 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58edee-45a9-4c11-8994-85c31a769de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:37.851575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58edee-45a9-4c11-8994-85c31a769de0 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:37.852216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc58edee-45a9-4c11-8994-85c31a769de0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:37.853641 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc58edee-45a9-4c11-8994-85c31a769de0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:37.861470 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c8c50-5d33-4c40-a4ec-d599a0fef9d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:37.862949 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c8c50-5d33-4c40-a4ec-d599a0fef9d5 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:37.863864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:37.974827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 114/223] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Sep 2 09:13:37 2022] POST /identity/v3/users => generated 297 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:13:37.975019 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 110/224] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:13:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:13:37.975113 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58edee-45a9-4c11-8994-85c31a769de0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:37.975218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0edf7-43e8-4ea3-ac9c-6af2f0559a97 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:37.975218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0edf7-43e8-4ea3-ac9c-6af2f0559a97 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:37.975218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c0edf7-43e8-4ea3-ac9c-6af2f0559a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:37.985174 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-83c0edf7-43e8-4ea3-ac9c-6af2f0559a97 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:37.986398 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-83c0edf7-43e8-4ea3-ac9c-6af2f0559a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:37.997668 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:38.075158 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 111/225] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:38.076089 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c0edf7-43e8-4ea3-ac9c-6af2f0559a97 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:38.076743 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58edee-45a9-4c11-8994-85c31a769de0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:38.118881 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0d5acb-61c7-4976-ad40-901c124b0d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:38.119834 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d5acb-61c7-4976-ad40-901c124b0d79 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:38.120273 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d5acb-61c7-4976-ad40-901c124b0d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:38.120550 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d5acb-61c7-4976-ad40-901c124b0d79 None admin] PATH_INFO: `/v3/projects/dac3a1a96e914a549133e73f03943618` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:38.122858 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0d5acb-61c7-4976-ad40-901c124b0d79 None admin] RBAC: Authorizing `identity:get_project(project_id=dac3a1a96e914a549133e73f03943618)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:38.124200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0d5acb-61c7-4976-ad40-901c124b0d79 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:38.125843 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 115/226] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:37 2022] GET /identity/v3/projects/dac3a1a96e914a549133e73f03943618 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:38.131895 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0d5acb-61c7-4976-ad40-901c124b0d79 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:38.211302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:38.222444 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c0edf7-43e8-4ea3-ac9c-6af2f0559a97 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:38.265783 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0d5acb-61c7-4976-ad40-901c124b0d79 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:38.268783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e14bc7-4f25-4362-b0e9-772cb390f98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:38.269433 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e14bc7-4f25-4362-b0e9-772cb390f98a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:38.269711 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e14bc7-4f25-4362-b0e9-772cb390f98a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:38.269982 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e14bc7-4f25-4362-b0e9-772cb390f98a None admin] PATH_INFO: `/v3/projects/dac3a1a96e914a549133e73f03943618` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:38.271148 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e14bc7-4f25-4362-b0e9-772cb390f98a None admin] RBAC: Authorizing `identity:get_project(project_id=dac3a1a96e914a549133e73f03943618)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:38.272592 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e14bc7-4f25-4362-b0e9-772cb390f98a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:38.274214 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 112/227] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:38 2022] GET /identity/v3/projects/dac3a1a96e914a549133e73f03943618 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:38.280893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e14bc7-4f25-4362-b0e9-772cb390f98a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:38.299070 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:38.308786 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09109cf7-2a05-4d3c-8cae-6f293f065a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:38.309852 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09109cf7-2a05-4d3c-8cae-6f293f065a59 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:38.313376 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09109cf7-2a05-4d3c-8cae-6f293f065a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:38.313376 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09109cf7-2a05-4d3c-8cae-6f293f065a59 None admin] PATH_INFO: `/v3/projects/dac3a1a96e914a549133e73f03943618/users/941ab0618914451992a54b0b138a7a70/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:38.313376 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09109cf7-2a05-4d3c-8cae-6f293f065a59 None admin] RBAC: Authorizing `identity:create_grant(project_id=dac3a1a96e914a549133e73f03943618, user_id=941ab0618914451992a54b0b138a7a70, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:38.314758 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09109cf7-2a05-4d3c-8cae-6f293f065a59 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:38.389541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 116/228] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Sep 2 09:13:38 2022] PUT /identity/v3/projects/dac3a1a96e914a549133e73f03943618/users/941ab0618914451992a54b0b138a7a70/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:13:38.390400 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09109cf7-2a05-4d3c-8cae-6f293f065a59 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:38.426935 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e14bc7-4f25-4362-b0e9-772cb390f98a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:38.497055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10c426-29f5-46db-9d92-5007b7b7a190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:38.497791 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10c426-29f5-46db-9d92-5007b7b7a190 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:38.498080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10c426-29f5-46db-9d92-5007b7b7a190 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:38.498356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10c426-29f5-46db-9d92-5007b7b7a190 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:38.499169 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10c426-29f5-46db-9d92-5007b7b7a190 None admin] RBAC: Adding query filter params (role.id=efe6bdd5112340b48b262ceafbc19c04, scope.project.id=dac3a1a96e914a549133e73f03943618, user.id=b28f835d8ca443768cc83703b41ae12b) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:38.499758 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10c426-29f5-46db-9d92-5007b7b7a190 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:38.501103 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10c426-29f5-46db-9d92-5007b7b7a190 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:38.507793 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 113/229] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:13:38 2022] GET /identity/v3/role_assignments?user.id=b28f835d8ca443768cc83703b41ae12b&scope.project.id=dac3a1a96e914a549133e73f03943618&role.id=efe6bdd5112340b48b262ceafbc19c04 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:13:38.582573 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09109cf7-2a05-4d3c-8cae-6f293f065a59 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:38.612187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:38.624779 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2cfa7-791c-474c-a6c0-2ddebc9afc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:38.625528 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2cfa7-791c-474c-a6c0-2ddebc9afc93 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:38.625824 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2cfa7-791c-474c-a6c0-2ddebc9afc93 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:38.626235 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c2cfa7-791c-474c-a6c0-2ddebc9afc93 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:38.641304 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c2cfa7-791c-474c-a6c0-2ddebc9afc93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:38.646390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-37c2cfa7-791c-474c-a6c0-2ddebc9afc93 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 09:13:38.646390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:38.646390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:38.646390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:38.646390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:38.646390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:38.646390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:38.646390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:38.646390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:38.646390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:38.646390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:38.646390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:38.646390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:38.646390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:38.646390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:38.646390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:38.647340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:38.647340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:38.647340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:38.647340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:38.647340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:38.647340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:38.647340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:38.647340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:38.647340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:38.647340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:38.647340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:38.647340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:38.647340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:38.647340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:38.647340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:38.647340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:38.648117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:38.648117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:38.648117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:38.648117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:38.648117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:38.648117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:38.648117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:38.648117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:38.648117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:38.648117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:38.648117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 09:13:38.648117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:13:38.648117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 117/230] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Sep 2 09:13:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:38.656571 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10c426-29f5-46db-9d92-5007b7b7a190 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:38.771405 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:38.794862 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10c426-29f5-46db-9d92-5007b7b7a190 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:38.840888 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747e470-e39f-4b09-b8c8-af89a717ae92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:38.841426 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747e470-e39f-4b09-b8c8-af89a717ae92 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:38.841704 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747e470-e39f-4b09-b8c8-af89a717ae92 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:38.841977 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747e470-e39f-4b09-b8c8-af89a717ae92 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:38.842649 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747e470-e39f-4b09-b8c8-af89a717ae92 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:38.842978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747e470-e39f-4b09-b8c8-af89a717ae92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:38.844673 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747e470-e39f-4b09-b8c8-af89a717ae92 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:38.857106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 114/231] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:13:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:38.863793 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2cfa7-791c-474c-a6c0-2ddebc9afc93 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:38.976485 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c2cfa7-791c-474c-a6c0-2ddebc9afc93 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:39.012556 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d5b280-1cd5-4d7e-8f7b-09ccb875b62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:39.013236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5b280-1cd5-4d7e-8f7b-09ccb875b62f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:39.013520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5b280-1cd5-4d7e-8f7b-09ccb875b62f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:39.013797 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5b280-1cd5-4d7e-8f7b-09ccb875b62f None admin] PATH_INFO: `/v3/users/941ab0618914451992a54b0b138a7a70` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:39.014954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d5b280-1cd5-4d7e-8f7b-09ccb875b62f None admin] RBAC: Authorizing `identity:get_user(user_id=941ab0618914451992a54b0b138a7a70)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:39.016407 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d5b280-1cd5-4d7e-8f7b-09ccb875b62f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:39.017987 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 118/232] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:38 2022] GET /identity/v3/users/941ab0618914451992a54b0b138a7a70 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:39.023548 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747e470-e39f-4b09-b8c8-af89a717ae92 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:39.146506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747e470-e39f-4b09-b8c8-af89a717ae92 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:39.186905 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9869f1-b979-4185-bf90-3786fe4688dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:39.187546 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9869f1-b979-4185-bf90-3786fe4688dc None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:39.187826 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9869f1-b979-4185-bf90-3786fe4688dc None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:39.188095 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9869f1-b979-4185-bf90-3786fe4688dc None admin] PATH_INFO: `/v3/projects/dac3a1a96e914a549133e73f03943618` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:39.189215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9869f1-b979-4185-bf90-3786fe4688dc None admin] RBAC: Authorizing `identity:get_project(project_id=dac3a1a96e914a549133e73f03943618)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:39.190655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9869f1-b979-4185-bf90-3786fe4688dc None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:39.192303 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 115/233] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:39 2022] GET /identity/v3/projects/dac3a1a96e914a549133e73f03943618 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:39.198445 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d5b280-1cd5-4d7e-8f7b-09ccb875b62f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:39.250084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:39.298924 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d5b280-1cd5-4d7e-8f7b-09ccb875b62f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:39.333895 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5786e9-b10b-43a8-a8f0-bd909ff23b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:39.334559 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5786e9-b10b-43a8-a8f0-bd909ff23b07 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:39.334883 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5786e9-b10b-43a8-a8f0-bd909ff23b07 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:39.335157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5786e9-b10b-43a8-a8f0-bd909ff23b07 None admin] PATH_INFO: `/v3/projects/dac3a1a96e914a549133e73f03943618/users/941ab0618914451992a54b0b138a7a70/roles/f2fe9fb31024471e9e2bea2b12715de6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:39.337341 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5786e9-b10b-43a8-a8f0-bd909ff23b07 None admin] RBAC: Authorizing `identity:create_grant(project_id=dac3a1a96e914a549133e73f03943618, user_id=941ab0618914451992a54b0b138a7a70, role_id=f2fe9fb31024471e9e2bea2b12715de6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:39.338830 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5786e9-b10b-43a8-a8f0-bd909ff23b07 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:39.444860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 119/234] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Sep 2 09:13:39 2022] PUT /identity/v3/projects/dac3a1a96e914a549133e73f03943618/users/941ab0618914451992a54b0b138a7a70/roles/f2fe9fb31024471e9e2bea2b12715de6 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:13:40.544009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ead3b3-f1a4-442a-b71b-564f57b23779 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:40.544552 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ead3b3-f1a4-442a-b71b-564f57b23779 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:40.544552 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ead3b3-f1a4-442a-b71b-564f57b23779 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:40.546153 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 116/235] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:40.551098 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f873fb2-997c-45f2-bd12-147f9b5bfa7c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:40.551503 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f873fb2-997c-45f2-bd12-147f9b5bfa7c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:40.551668 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f873fb2-997c-45f2-bd12-147f9b5bfa7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:40.570197 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8f873fb2-997c-45f2-bd12-147f9b5bfa7c None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:40.571432 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8f873fb2-997c-45f2-bd12-147f9b5bfa7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:40.612793 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e206a9a-717c-49b2-ad87-d2725e7f2c67 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:40.613125 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e206a9a-717c-49b2-ad87-d2725e7f2c67 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:40.613343 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e206a9a-717c-49b2-ad87-d2725e7f2c67 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:40.614935 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 117/236] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:13:40.620176 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccad6c4-237c-42ab-bf92-595849227053 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:40.620732 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccad6c4-237c-42ab-bf92-595849227053 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:40.621225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccad6c4-237c-42ab-bf92-595849227053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:40.643148 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:40.652485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8ccad6c4-237c-42ab-bf92-595849227053 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:40.654785 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8ccad6c4-237c-42ab-bf92-595849227053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:40.682833 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 120/237] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:40.694062 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3f984-946e-4ae6-b356-052b297402f4 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:40.697329 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3f984-946e-4ae6-b356-052b297402f4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:40.698322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3f984-946e-4ae6-b356-052b297402f4 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:40.699875 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 121/238] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:40.706260 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b0638-9ec5-4a0c-85ef-7f833f14ad83 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:40.708033 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b0638-9ec5-4a0c-85ef-7f833f14ad83 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:40.708515 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b0638-9ec5-4a0c-85ef-7f833f14ad83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:40.742101 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-484b0638-9ec5-4a0c-85ef-7f833f14ad83 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:40.745644 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-484b0638-9ec5-4a0c-85ef-7f833f14ad83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:40.791141 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 118/239] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 09:13:40.795104 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:40.840599 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 122/240] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 09:13:40.987269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c44df-4cca-40f2-9fbf-1fa09d707544 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:40.987555 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c44df-4cca-40f2-9fbf-1fa09d707544 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:40.987829 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35c44df-4cca-40f2-9fbf-1fa09d707544 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:40.989197 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 119/241] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:40.994274 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d6d09-c10e-4471-9d4e-ac798f1e1c67 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:40.995087 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d6d09-c10e-4471-9d4e-ac798f1e1c67 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:40.995370 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d6d09-c10e-4471-9d4e-ac798f1e1c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:41.023624 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f89d6d09-c10e-4471-9d4e-ac798f1e1c67 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:41.024809 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f89d6d09-c10e-4471-9d4e-ac798f1e1c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:41.065172 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c44df-4cca-40f2-9fbf-1fa09d707544 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:41.135788 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 123/242] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:41.163084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d6d09-c10e-4471-9d4e-ac798f1e1c67 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:41.206674 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35c44df-4cca-40f2-9fbf-1fa09d707544 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:41.242014 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b2c85-3379-499c-891e-93b8cd5c3e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:41.242760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b2c85-3379-499c-891e-93b8cd5c3e46 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:41.242942 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b2c85-3379-499c-891e-93b8cd5c3e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:41.243214 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b2c85-3379-499c-891e-93b8cd5c3e46 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:41.245823 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:41.254114 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4b2c85-3379-499c-891e-93b8cd5c3e46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:41.259055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-df4b2c85-3379-499c-891e-93b8cd5c3e46 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:13:41.259055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:41.259055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:41.259055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:41.259055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:41.259055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:41.259055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:41.259055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:41.259055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:41.259055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:41.259055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:41.259055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:41.259055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:41.259055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:41.259055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:41.259055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:41.261931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:41.261931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:41.261931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:41.261931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:41.261931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:41.261931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:41.261931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:41.261931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:41.261931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:41.261931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:41.261931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:41.261931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:41.261931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:41.261931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:41.261931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:41.261931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:41.262750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:41.262750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:41.262750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:41.262750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:41.262750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:41.262750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:41.262750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:41.262750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:41.262750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:41.262750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:41.262750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:13:41.262750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:13:41.262750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 120/243] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:13:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:41.262750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b2c85-3379-499c-891e-93b8cd5c3e46 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:41.281762 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:41.285651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d6d09-c10e-4471-9d4e-ac798f1e1c67 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:41.324652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325bae1a-91ac-4eb9-a6e9-bbbb9bf34f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:41.325611 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bae1a-91ac-4eb9-a6e9-bbbb9bf34f53 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:41.326340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bae1a-91ac-4eb9-a6e9-bbbb9bf34f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:41.326844 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bae1a-91ac-4eb9-a6e9-bbbb9bf34f53 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:41.328261 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325bae1a-91ac-4eb9-a6e9-bbbb9bf34f53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:41.330056 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325bae1a-91ac-4eb9-a6e9-bbbb9bf34f53 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:41.331982 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 124/244] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:13:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:13:41.332946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325bae1a-91ac-4eb9-a6e9-bbbb9bf34f53 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:41.369788 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b2c85-3379-499c-891e-93b8cd5c3e46 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:41.406209 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b08a7-fe98-4d8f-a80e-99644e93f5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:41.406898 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b08a7-fe98-4d8f-a80e-99644e93f5bb None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:41.407180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b08a7-fe98-4d8f-a80e-99644e93f5bb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:41.407496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b08a7-fe98-4d8f-a80e-99644e93f5bb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:41.418994 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9b08a7-fe98-4d8f-a80e-99644e93f5bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:41.425648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-5c9b08a7-fe98-4d8f-a80e-99644e93f5bb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 09:13:41.425648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:41.425648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:41.425648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:41.425648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:41.425648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:41.425648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:41.425648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:41.425648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:41.425648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:41.425648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:41.425648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:41.425648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:41.425648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:41.425648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:41.425648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:41.426519 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:41.426519 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:41.426519 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:41.426519 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:41.426519 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:41.426519 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:41.426519 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:41.426519 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:41.426519 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:41.426519 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:41.426519 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:41.426519 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:41.426519 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:41.426519 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:41.426519 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:41.426519 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:41.427684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:41.427684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:41.427684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:41.427684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:41.427684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:41.427684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:41.427684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:41.427684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:41.427684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:41.427684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:41.427684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 09:13:41.427684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:13:41.428377 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 121/245] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:41.428804 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b08a7-fe98-4d8f-a80e-99644e93f5bb None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:41.445892 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325bae1a-91ac-4eb9-a6e9-bbbb9bf34f53 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:41.482031 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c9422-79f8-4302-852a-aefae51e0b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:41.482641 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c9422-79f8-4302-852a-aefae51e0b5f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:41.482948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c9422-79f8-4302-852a-aefae51e0b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:41.483218 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c9422-79f8-4302-852a-aefae51e0b5f None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:41.483891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c9422-79f8-4302-852a-aefae51e0b5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:41.484194 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c9422-79f8-4302-852a-aefae51e0b5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:41.485551 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c9422-79f8-4302-852a-aefae51e0b5f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:41.492206 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:41.494224 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 125/246] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:41.498675 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c9422-79f8-4302-852a-aefae51e0b5f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:41.527881 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:41.537294 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b08a7-fe98-4d8f-a80e-99644e93f5bb None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:41.577840 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6d913-2762-413c-a6a6-6b5479d2dff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:41.578483 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6d913-2762-413c-a6a6-6b5479d2dff9 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:41.578738 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6d913-2762-413c-a6a6-6b5479d2dff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:41.578973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c6d913-2762-413c-a6a6-6b5479d2dff9 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:41.579622 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6d913-2762-413c-a6a6-6b5479d2dff9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:41.581066 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c6d913-2762-413c-a6a6-6b5479d2dff9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:41.603351 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c9422-79f8-4302-852a-aefae51e0b5f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:41.636317 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f92c89-45df-4598-b4b0-1c30f3637fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:41.636921 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f92c89-45df-4598-b4b0-1c30f3637fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:41.637195 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f92c89-45df-4598-b4b0-1c30f3637fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:41.637462 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f92c89-45df-4598-b4b0-1c30f3637fd3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:41.647441 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f92c89-45df-4598-b4b0-1c30f3637fd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:41.652916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-17f92c89-45df-4598-b4b0-1c30f3637fd3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:41.652916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:41.652916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:41.652916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:41.652916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:41.652916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:41.652916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:41.652916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:41.652916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:41.652916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:41.652916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:41.652916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:41.652916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:41.652916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:41.652916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:41.652916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:41.653827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:41.653827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:41.653827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:41.653827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:41.653827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:41.653827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:41.653827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:41.653827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:41.653827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:41.653827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:41.653827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:41.653827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:41.653827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:41.653827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:41.653827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:41.653827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:41.654608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:41.654608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:41.654608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:41.654608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:41.654608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:41.654608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:41.654608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:41.654608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:41.654608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:41.654608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:41.654608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:41.654608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:13:41.654608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 126/247] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:41.657418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f92c89-45df-4598-b4b0-1c30f3637fd3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:41.672956 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 122/248] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Sep 2 09:13:41 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:13:41.672956 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6d913-2762-413c-a6a6-6b5479d2dff9 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:41.754585 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:41.765282 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f92c89-45df-4598-b4b0-1c30f3637fd3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:41.803659 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49badd3-0443-4c6c-b081-9c27038e2c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:41.804442 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49badd3-0443-4c6c-b081-9c27038e2c1f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:41.804723 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49badd3-0443-4c6c-b081-9c27038e2c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:41.804995 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49badd3-0443-4c6c-b081-9c27038e2c1f None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:41.805674 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49badd3-0443-4c6c-b081-9c27038e2c1f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:41.805975 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49badd3-0443-4c6c-b081-9c27038e2c1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:41.807879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c6d913-2762-413c-a6a6-6b5479d2dff9 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:41.807987 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49badd3-0443-4c6c-b081-9c27038e2c1f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:41.818345 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 127/249] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Sep 2 09:13:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:41.820036 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49badd3-0443-4c6c-b081-9c27038e2c1f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:41.823854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:41.851478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b34063-9791-4ed6-961a-b25927405547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:41.852307 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b34063-9791-4ed6-961a-b25927405547 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:41.852726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b34063-9791-4ed6-961a-b25927405547 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:41.853156 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b34063-9791-4ed6-961a-b25927405547 None admin] PATH_INFO: `/v3/users/941ab0618914451992a54b0b138a7a70` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:41.854818 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b34063-9791-4ed6-961a-b25927405547 None admin] RBAC: Authorizing `identity:get_user(user_id=941ab0618914451992a54b0b138a7a70)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:41.856338 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b34063-9791-4ed6-961a-b25927405547 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:41.857943 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 123/250] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:41 2022] GET /identity/v3/users/941ab0618914451992a54b0b138a7a70 => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:41.859204 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cfcd04-47bc-455b-b306-4068893485b3 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:41.859488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cfcd04-47bc-455b-b306-4068893485b3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:41.859744 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cfcd04-47bc-455b-b306-4068893485b3 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:41.861115 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 124/251] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:41.865917 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cfcd04-47bc-455b-b306-4068893485b3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:41.955781 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49badd3-0443-4c6c-b081-9c27038e2c1f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:41.991994 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cfcd04-47bc-455b-b306-4068893485b3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:41.994448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558c00f-9b5c-4dc4-b303-d730ea688752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:41.995676 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558c00f-9b5c-4dc4-b303-d730ea688752 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:41.995963 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558c00f-9b5c-4dc4-b303-d730ea688752 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:41.996217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1558c00f-9b5c-4dc4-b303-d730ea688752 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:41.996881 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1558c00f-9b5c-4dc4-b303-d730ea688752 None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:41.997181 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1558c00f-9b5c-4dc4-b303-d730ea688752 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:41.998614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1558c00f-9b5c-4dc4-b303-d730ea688752 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:42.011116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 128/252] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Sep 2 09:13:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:42.011956 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558c00f-9b5c-4dc4-b303-d730ea688752 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:42.030595 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:42.035808 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3134d65-ed68-45c6-b536-e655da7e3129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:42.036465 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3134d65-ed68-45c6-b536-e655da7e3129 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:42.036747 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3134d65-ed68-45c6-b536-e655da7e3129 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:42.037016 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3134d65-ed68-45c6-b536-e655da7e3129 None admin] PATH_INFO: `/v3/users/6845b0bb33f04b45b9ee70ce3b65db01` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:42.053889 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3134d65-ed68-45c6-b536-e655da7e3129 None admin] RBAC: Authorizing `identity:get_user(user_id=6845b0bb33f04b45b9ee70ce3b65db01)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:42.055594 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3134d65-ed68-45c6-b536-e655da7e3129 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:42.057794 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 125/253] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:41 2022] GET /identity/v3/users/6845b0bb33f04b45b9ee70ce3b65db01 => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:42.067104 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a9a07-ecc7-4bba-a0d5-8b414dbce103 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:42.067408 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a9a07-ecc7-4bba-a0d5-8b414dbce103 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:42.067917 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a9a07-ecc7-4bba-a0d5-8b414dbce103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:42.088540 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d73a9a07-ecc7-4bba-a0d5-8b414dbce103 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:42.089750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d73a9a07-ecc7-4bba-a0d5-8b414dbce103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:42.122361 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:42.141592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1558c00f-9b5c-4dc4-b303-d730ea688752 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:42.171290 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 126/254] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:42.173307 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a9a07-ecc7-4bba-a0d5-8b414dbce103 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:42.179865 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562d6cb-ab46-477a-b642-facfe699d995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:42.180495 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562d6cb-ab46-477a-b642-facfe699d995 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:42.180947 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562d6cb-ab46-477a-b642-facfe699d995 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:42.181226 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562d6cb-ab46-477a-b642-facfe699d995 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:42.279587 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a9a07-ecc7-4bba-a0d5-8b414dbce103 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:42.304796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562d6cb-ab46-477a-b642-facfe699d995 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:42.306279 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562d6cb-ab46-477a-b642-facfe699d995 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:42.323712 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ae4b0-ea95-41a3-a14c-db684ab64180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:42.324764 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ae4b0-ea95-41a3-a14c-db684ab64180 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:42.325230 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:42.326332 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ae4b0-ea95-41a3-a14c-db684ab64180 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:42.326904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871ae4b0-ea95-41a3-a14c-db684ab64180 None admin] PATH_INFO: `/v3/users/b28f835d8ca443768cc83703b41ae12b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:42.328248 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871ae4b0-ea95-41a3-a14c-db684ab64180 None admin] RBAC: Authorizing `identity:get_user(user_id=b28f835d8ca443768cc83703b41ae12b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:42.329888 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871ae4b0-ea95-41a3-a14c-db684ab64180 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:42.331719 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 127/255] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:42 2022] GET /identity/v3/users/b28f835d8ca443768cc83703b41ae12b => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:42.337193 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ae4b0-ea95-41a3-a14c-db684ab64180 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:42.397893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:42.459763 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 129/256] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:13:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:13:42.460626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562d6cb-ab46-477a-b642-facfe699d995 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:42.496561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871ae4b0-ea95-41a3-a14c-db684ab64180 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:42.540610 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2e199-8f20-416e-9c69-5a126679e21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:42.541232 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2e199-8f20-416e-9c69-5a126679e21b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:42.542127 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2e199-8f20-416e-9c69-5a126679e21b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:42.542402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2e199-8f20-416e-9c69-5a126679e21b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:42.568170 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562d6cb-ab46-477a-b642-facfe699d995 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:42.583236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:42.605367 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfdf06c-ee7d-44dc-8108-62018462c1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:42.606054 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfdf06c-ee7d-44dc-8108-62018462c1bc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:42.606269 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfdf06c-ee7d-44dc-8108-62018462c1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:42.606537 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfdf06c-ee7d-44dc-8108-62018462c1bc None admin] PATH_INFO: `/v3/projects/dac3a1a96e914a549133e73f03943618` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:42.608678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfdf06c-ee7d-44dc-8108-62018462c1bc None admin] RBAC: Authorizing `identity:get_project(project_id=dac3a1a96e914a549133e73f03943618)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:42.609114 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfdf06c-ee7d-44dc-8108-62018462c1bc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:42.615477 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 130/257] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:42 2022] GET /identity/v3/projects/dac3a1a96e914a549133e73f03943618 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:42.615477 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfdf06c-ee7d-44dc-8108-62018462c1bc None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:42.694823 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2e199-8f20-416e-9c69-5a126679e21b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:42.696091 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2e199-8f20-416e-9c69-5a126679e21b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:42.731439 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:42.733885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfdf06c-ee7d-44dc-8108-62018462c1bc None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:42.770505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d59c0-fae5-4d67-bf6b-5508f28a0d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:42.771392 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d59c0-fae5-4d67-bf6b-5508f28a0d1b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:42.771840 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d59c0-fae5-4d67-bf6b-5508f28a0d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:42.772251 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808d59c0-fae5-4d67-bf6b-5508f28a0d1b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:42.784083 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808d59c0-fae5-4d67-bf6b-5508f28a0d1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:42.789978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-808d59c0-fae5-4d67-bf6b-5508f28a0d1b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 09:13:42.789978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:42.789978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:42.789978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:42.789978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:42.789978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:42.789978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:42.789978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:42.789978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:42.789978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:42.789978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:42.789978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:42.789978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:42.789978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:42.789978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:42.789978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:42.791075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:42.791075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:42.791075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:42.791075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:42.791075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:42.791075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:42.791075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:42.791075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:42.791075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:42.791075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:42.791075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:42.791075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:42.791075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:42.791075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:42.791075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:42.791075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:42.792777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:42.792777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:42.792777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:42.792777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:42.792777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:42.792777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:42.792777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:42.792777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:42.792777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:42.792777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:42.792777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 09:13:42.792777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:13:42.794696 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 131/258] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Sep 2 09:13:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:42.795737 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d59c0-fae5-4d67-bf6b-5508f28a0d1b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:42.806759 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 128/259] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:13:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:13:42.806759 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2e199-8f20-416e-9c69-5a126679e21b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:42.846243 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:42.910434 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2e199-8f20-416e-9c69-5a126679e21b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:42.910835 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808d59c0-fae5-4d67-bf6b-5508f28a0d1b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:42.945016 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07041d7-ccf2-42ef-9693-df7ebaf25c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:42.945601 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07041d7-ccf2-42ef-9693-df7ebaf25c4b None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:42.945887 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07041d7-ccf2-42ef-9693-df7ebaf25c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:42.946159 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07041d7-ccf2-42ef-9693-df7ebaf25c4b None admin] PATH_INFO: `/v3/projects/dac3a1a96e914a549133e73f03943618/users/b28f835d8ca443768cc83703b41ae12b/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:42.947302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106804c-b55b-4b8c-adf5-1bd472dcb359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:42.948587 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106804c-b55b-4b8c-adf5-1bd472dcb359 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:42.948691 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07041d7-ccf2-42ef-9693-df7ebaf25c4b None admin] RBAC: Authorizing `identity:create_grant(project_id=dac3a1a96e914a549133e73f03943618, user_id=b28f835d8ca443768cc83703b41ae12b, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:42.948878 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106804c-b55b-4b8c-adf5-1bd472dcb359 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:42.949142 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7106804c-b55b-4b8c-adf5-1bd472dcb359 None admin] PATH_INFO: `/v3/projects/dac3a1a96e914a549133e73f03943618` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:42.950086 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07041d7-ccf2-42ef-9693-df7ebaf25c4b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:42.950275 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106804c-b55b-4b8c-adf5-1bd472dcb359 None admin] RBAC: Authorizing `identity:get_project(project_id=dac3a1a96e914a549133e73f03943618)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:42.952064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7106804c-b55b-4b8c-adf5-1bd472dcb359 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:42.953567 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:42.954942 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 132/260] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:42 2022] GET /identity/v3/projects/dac3a1a96e914a549133e73f03943618 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:42.955759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106804c-b55b-4b8c-adf5-1bd472dcb359 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:43.046543 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 129/261] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Sep 2 09:13:42 2022] PUT /identity/v3/projects/dac3a1a96e914a549133e73f03943618/users/b28f835d8ca443768cc83703b41ae12b/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:13:43.052780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07041d7-ccf2-42ef-9693-df7ebaf25c4b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:43.088881 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7106804c-b55b-4b8c-adf5-1bd472dcb359 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:43.104514 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:43.148520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4fe54-1e03-4f7a-8497-b0b610136ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:43.149256 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4fe54-1e03-4f7a-8497-b0b610136ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:43.149540 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4fe54-1e03-4f7a-8497-b0b610136ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:43.149817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d4fe54-1e03-4f7a-8497-b0b610136ab6 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:43.150593 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4fe54-1e03-4f7a-8497-b0b610136ab6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:43.151494 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4fe54-1e03-4f7a-8497-b0b610136ab6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:43.154195 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d4fe54-1e03-4f7a-8497-b0b610136ab6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:43.173021 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 133/262] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:13:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:43.173919 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4fe54-1e03-4f7a-8497-b0b610136ab6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:43.257075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07041d7-ccf2-42ef-9693-df7ebaf25c4b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:43.296594 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca6c25-0a0f-4163-8826-e1262e6ec48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:43.297300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca6c25-0a0f-4163-8826-e1262e6ec48c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:43.297580 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca6c25-0a0f-4163-8826-e1262e6ec48c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:43.297854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bca6c25-0a0f-4163-8826-e1262e6ec48c None admin] PATH_INFO: `/v3/projects/dac3a1a96e914a549133e73f03943618` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:43.299069 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca6c25-0a0f-4163-8826-e1262e6ec48c None admin] RBAC: Authorizing `identity:get_project(project_id=dac3a1a96e914a549133e73f03943618)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:43.300599 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bca6c25-0a0f-4163-8826-e1262e6ec48c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:43.302559 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 130/263] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:43 2022] GET /identity/v3/projects/dac3a1a96e914a549133e73f03943618 => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:43.303420 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca6c25-0a0f-4163-8826-e1262e6ec48c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:43.330108 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d4fe54-1e03-4f7a-8497-b0b610136ab6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:43.373169 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985fa704-0c17-4d91-a900-f4452d5ce4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:43.373788 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fa704-0c17-4d91-a900-f4452d5ce4f4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:43.374070 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fa704-0c17-4d91-a900-f4452d5ce4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:43.374340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fa704-0c17-4d91-a900-f4452d5ce4f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:43.375167 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985fa704-0c17-4d91-a900-f4452d5ce4f4 None admin] RBAC: Adding query filter params (role.id=f886b8e8889549e896a8fb3426cce9e1, scope.project.id=dac3a1a96e914a549133e73f03943618, user.id=941ab0618914451992a54b0b138a7a70) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:43.375535 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985fa704-0c17-4d91-a900-f4452d5ce4f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:43.377031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:43.377168 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985fa704-0c17-4d91-a900-f4452d5ce4f4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:43.383680 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 134/264] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:13:43 2022] GET /identity/v3/role_assignments?user.id=941ab0618914451992a54b0b138a7a70&scope.project.id=dac3a1a96e914a549133e73f03943618&role.id=f886b8e8889549e896a8fb3426cce9e1 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:43.384215 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985fa704-0c17-4d91-a900-f4452d5ce4f4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:43.427723 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bca6c25-0a0f-4163-8826-e1262e6ec48c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:43.459710 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:43.479454 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76958cd-0960-4b85-bf5e-3c23403f68e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:43.480347 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76958cd-0960-4b85-bf5e-3c23403f68e0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:43.480766 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76958cd-0960-4b85-bf5e-3c23403f68e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:43.481424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76958cd-0960-4b85-bf5e-3c23403f68e0 None admin] PATH_INFO: `/v3/users/941ab0618914451992a54b0b138a7a70` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:43.484366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76958cd-0960-4b85-bf5e-3c23403f68e0 None admin] RBAC: Authorizing `identity:get_user(user_id=941ab0618914451992a54b0b138a7a70)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:43.486036 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76958cd-0960-4b85-bf5e-3c23403f68e0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:43.490578 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 131/265] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:43 2022] GET /identity/v3/users/941ab0618914451992a54b0b138a7a70 => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:43.496399 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76958cd-0960-4b85-bf5e-3c23403f68e0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:43.628878 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985fa704-0c17-4d91-a900-f4452d5ce4f4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:43.666834 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76958cd-0960-4b85-bf5e-3c23403f68e0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:43.701624 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:43.711554 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0ddf1-c18d-4c0f-86c8-fb26c1156fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:43.714667 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0ddf1-c18d-4c0f-86c8-fb26c1156fda None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:43.714667 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0ddf1-c18d-4c0f-86c8-fb26c1156fda None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:43.714667 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0ddf1-c18d-4c0f-86c8-fb26c1156fda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:43.718007 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8b952-20ac-4cac-9e78-50529ebb90f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:43.718630 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8b952-20ac-4cac-9e78-50529ebb90f6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:43.718938 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8b952-20ac-4cac-9e78-50529ebb90f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:43.719205 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8b952-20ac-4cac-9e78-50529ebb90f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:43.719931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8b952-20ac-4cac-9e78-50529ebb90f6 None admin] RBAC: Adding query filter params (role.id=b381d11646e54d1197676600202a6144, scope.project.id=dac3a1a96e914a549133e73f03943618, user.id=6845b0bb33f04b45b9ee70ce3b65db01) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:43.721222 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8b952-20ac-4cac-9e78-50529ebb90f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:43.722659 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8b952-20ac-4cac-9e78-50529ebb90f6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:43.737779 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 135/266] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:13:43 2022] GET /identity/v3/role_assignments?user.id=6845b0bb33f04b45b9ee70ce3b65db01&scope.project.id=dac3a1a96e914a549133e73f03943618&role.id=b381d11646e54d1197676600202a6144 => generated 259 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:43.818601 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0ddf1-c18d-4c0f-86c8-fb26c1156fda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:43.820185 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0ddf1-c18d-4c0f-86c8-fb26c1156fda None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:43.958688 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c6496-b83a-43d7-831a-7dbd91309027 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:43.959053 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c6496-b83a-43d7-831a-7dbd91309027 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:43.959245 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c6496-b83a-43d7-831a-7dbd91309027 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:43.960624 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 136/267] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:13:43.966102 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31cce17-8c86-486a-8687-b9f242e3f527 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:43.966370 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31cce17-8c86-486a-8687-b9f242e3f527 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:43.966643 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31cce17-8c86-486a-8687-b9f242e3f527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:43.996303 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 132/268] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:13:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:13:43.997324 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f31cce17-8c86-486a-8687-b9f242e3f527 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:43.998559 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f31cce17-8c86-486a-8687-b9f242e3f527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:44.003448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0ddf1-c18d-4c0f-86c8-fb26c1156fda None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:44.061304 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:44.146694 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 137/269] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 09:13:44.176265 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0ddf1-c18d-4c0f-86c8-fb26c1156fda None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:44.217219 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9364ba-7715-4949-9714-b5d402d037d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:44.217773 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9364ba-7715-4949-9714-b5d402d037d3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:44.218048 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9364ba-7715-4949-9714-b5d402d037d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:44.218320 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9364ba-7715-4949-9714-b5d402d037d3 None admin] PATH_INFO: `/v3/projects/dac3a1a96e914a549133e73f03943618` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:44.219483 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9364ba-7715-4949-9714-b5d402d037d3 None admin] RBAC: Authorizing `identity:get_project(project_id=dac3a1a96e914a549133e73f03943618)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:44.220998 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9364ba-7715-4949-9714-b5d402d037d3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:44.222626 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 133/270] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:44 2022] GET /identity/v3/projects/dac3a1a96e914a549133e73f03943618 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:44.228604 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31cce17-8c86-486a-8687-b9f242e3f527 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:44.249652 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028cbd1-6442-46b3-bb5e-8284e5412656 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:44.249938 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028cbd1-6442-46b3-bb5e-8284e5412656 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:44.250206 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028cbd1-6442-46b3-bb5e-8284e5412656 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:44.251664 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 134/271] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:13:44.257121 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080aa4f0-3180-4f23-b4ba-5778e673f0af None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:44.257458 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080aa4f0-3180-4f23-b4ba-5778e673f0af None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:44.257748 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080aa4f0-3180-4f23-b4ba-5778e673f0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:44.273956 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:44.291095 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-080aa4f0-3180-4f23-b4ba-5778e673f0af None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:44.294742 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-080aa4f0-3180-4f23-b4ba-5778e673f0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:44.407349 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 135/272] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:44.421670 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31cce17-8c86-486a-8687-b9f242e3f527 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:44.473316 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b3cb3-84db-4bf4-b575-ff0e9664463b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:44.473850 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b3cb3-84db-4bf4-b575-ff0e9664463b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:44.474136 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b3cb3-84db-4bf4-b575-ff0e9664463b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:44.474410 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6b3cb3-84db-4bf4-b575-ff0e9664463b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:44.475189 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b3cb3-84db-4bf4-b575-ff0e9664463b None admin] RBAC: Adding query filter params (role.id=f2fe9fb31024471e9e2bea2b12715de6, scope.project.id=dac3a1a96e914a549133e73f03943618, user.id=941ab0618914451992a54b0b138a7a70) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:44.475500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b3cb3-84db-4bf4-b575-ff0e9664463b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:44.476930 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6b3cb3-84db-4bf4-b575-ff0e9664463b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:44.484897 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 138/273] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:13:44 2022] GET /identity/v3/role_assignments?user.id=941ab0618914451992a54b0b138a7a70&scope.project.id=dac3a1a96e914a549133e73f03943618&role.id=f2fe9fb31024471e9e2bea2b12715de6 => generated 608 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:44.561386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080aa4f0-3180-4f23-b4ba-5778e673f0af None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:44.595793 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:44.680991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080aa4f0-3180-4f23-b4ba-5778e673f0af None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:44.722916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f3ff3-96a8-49d6-858f-ad41d3e2e6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:44.724377 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f3ff3-96a8-49d6-858f-ad41d3e2e6a8 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:44.724851 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f3ff3-96a8-49d6-858f-ad41d3e2e6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:44.725351 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f3ff3-96a8-49d6-858f-ad41d3e2e6a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:44.728576 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b3cb3-84db-4bf4-b575-ff0e9664463b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:44.737031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0f3ff3-96a8-49d6-858f-ad41d3e2e6a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:44.742209 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-ee0f3ff3-96a8-49d6-858f-ad41d3e2e6a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:44.742209 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:44.742209 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:44.742209 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:44.742209 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:44.742209 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:44.742209 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:44.742209 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:44.742209 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:44.742209 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:44.742209 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:44.742209 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:44.742209 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:44.742209 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:44.742209 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:44.742209 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:44.743122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:44.743122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:44.743122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:44.743122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:44.743122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:44.743122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:44.743122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:44.743122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:44.743122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:44.743122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:44.743122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:44.743122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:44.743122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:44.743122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:44.743122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:44.743122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:44.743900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:44.743900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:44.743900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:44.743900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:44.743900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:44.743900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:44.743900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:44.743900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:44.743900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:44.743900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:44.743900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:44.743900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:13:44.746841 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 136/274] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:44.749984 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f3ff3-96a8-49d6-858f-ad41d3e2e6a8 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:44.852162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:44.857728 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:44.864812 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6b3cb3-84db-4bf4-b575-ff0e9664463b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:44.873328 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0f3ff3-96a8-49d6-858f-ad41d3e2e6a8 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:44.918060 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a336e-75df-401e-b696-ca41d99c3f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:44.918677 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a336e-75df-401e-b696-ca41d99c3f90 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:44.918951 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f9040-d5b4-4c9e-861d-7805f62f4c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:44.919064 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a336e-75df-401e-b696-ca41d99c3f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:44.919257 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3a336e-75df-401e-b696-ca41d99c3f90 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:44.919935 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3a336e-75df-401e-b696-ca41d99c3f90 None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:44.920237 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3a336e-75df-401e-b696-ca41d99c3f90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:44.920458 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f9040-d5b4-4c9e-861d-7805f62f4c37 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:44.920929 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f9040-d5b4-4c9e-861d-7805f62f4c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:44.921389 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f9040-d5b4-4c9e-861d-7805f62f4c37 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:44.921991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3a336e-75df-401e-b696-ca41d99c3f90 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:44.923523 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f9040-d5b4-4c9e-861d-7805f62f4c37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:44.925525 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f9040-d5b4-4c9e-861d-7805f62f4c37 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:44.927371 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 139/275] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:13:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:44.933701 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f9040-d5b4-4c9e-861d-7805f62f4c37 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:44.937608 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 137/276] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Sep 2 09:13:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:44.943147 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a336e-75df-401e-b696-ca41d99c3f90 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:45.060515 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f9040-d5b4-4c9e-861d-7805f62f4c37 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:45.067960 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3a336e-75df-401e-b696-ca41d99c3f90 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:45.097641 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea62d65-8757-4df8-a94e-9da8689c6d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:45.098724 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea62d65-8757-4df8-a94e-9da8689c6d74 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:45.099201 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea62d65-8757-4df8-a94e-9da8689c6d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:45.099711 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea62d65-8757-4df8-a94e-9da8689c6d74 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:45.100544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea62d65-8757-4df8-a94e-9da8689c6d74 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:45.102169 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea62d65-8757-4df8-a94e-9da8689c6d74 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:45.104792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf21b387-999c-4f74-9e59-f12e8f7e54c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:45.105420 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21b387-999c-4f74-9e59-f12e8f7e54c5 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:45.105701 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21b387-999c-4f74-9e59-f12e8f7e54c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:45.106062 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21b387-999c-4f74-9e59-f12e8f7e54c5 None admin] PATH_INFO: `/v3/users/334c10c6eb4c4b1a9da1ed17c075a943` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:45.121723 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf21b387-999c-4f74-9e59-f12e8f7e54c5 None admin] RBAC: Authorizing `identity:get_user(user_id=334c10c6eb4c4b1a9da1ed17c075a943)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:45.123439 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf21b387-999c-4f74-9e59-f12e8f7e54c5 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:45.125206 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 138/277] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:44 2022] GET /identity/v3/users/334c10c6eb4c4b1a9da1ed17c075a943 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:13:45.131413 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf21b387-999c-4f74-9e59-f12e8f7e54c5 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:45.144716 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:45.228285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 140/278] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Sep 2 09:13:44 2022] POST /identity/v3/users => generated 293 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:13:45.234748 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf21b387-999c-4f74-9e59-f12e8f7e54c5 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:45.274627 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c915c70-d00d-485b-8013-167003ab5442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:45.275613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c915c70-d00d-485b-8013-167003ab5442 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:45.276037 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c915c70-d00d-485b-8013-167003ab5442 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:45.276450 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c915c70-d00d-485b-8013-167003ab5442 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:45.277400 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c915c70-d00d-485b-8013-167003ab5442 None admin] RBAC: Adding query filter params (role.id=efe6bdd5112340b48b262ceafbc19c04, user.id=334c10c6eb4c4b1a9da1ed17c075a943, scope.system=all) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:45.277932 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c915c70-d00d-485b-8013-167003ab5442 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:45.279658 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c915c70-d00d-485b-8013-167003ab5442 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:45.286177 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 139/279] 10.222.0.57 () {56 vars in 1308 bytes} [Fri Sep 2 09:13:45 2022] GET /identity/v3/role_assignments?user.id=334c10c6eb4c4b1a9da1ed17c075a943&scope.system=all&role.id=efe6bdd5112340b48b262ceafbc19c04 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:13:45.379910 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403125e-0600-4a81-85d8-5d76b910e6d7 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:45.380206 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403125e-0600-4a81-85d8-5d76b910e6d7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:45.380480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403125e-0600-4a81-85d8-5d76b910e6d7 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:45.382043 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 141/280] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:45.387609 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519c959-b7b5-40b0-93bc-e045e9ba3e1f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:45.387885 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519c959-b7b5-40b0-93bc-e045e9ba3e1f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:45.388166 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519c959-b7b5-40b0-93bc-e045e9ba3e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:45.407942 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e519c959-b7b5-40b0-93bc-e045e9ba3e1f None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:45.409536 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e519c959-b7b5-40b0-93bc-e045e9ba3e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:45.486760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 140/281] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:45.759208 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403125e-0600-4a81-85d8-5d76b910e6d7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:45.784215 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:45.884402 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403125e-0600-4a81-85d8-5d76b910e6d7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:45.934749 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a0c04a-2eb0-4414-a9dc-e01271c7c39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:45.935395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0c04a-2eb0-4414-a9dc-e01271c7c39c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:45.935673 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0c04a-2eb0-4414-a9dc-e01271c7c39c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:45.935944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a0c04a-2eb0-4414-a9dc-e01271c7c39c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:45.951992 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a0c04a-2eb0-4414-a9dc-e01271c7c39c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:45.957223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-e3a0c04a-2eb0-4414-a9dc-e01271c7c39c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:45.957223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:45.957223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:45.957223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:45.957223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:45.957223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:45.957223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:45.957223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:45.957223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:45.957223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:45.957223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:45.957223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:45.957223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:45.957223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:45.957223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:45.957223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:45.958095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:45.958095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:45.958095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:45.958095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:45.958095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:45.958095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:45.958095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:45.958095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:45.958095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:45.958095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:45.958095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:45.958095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:45.958095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:45.958095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:45.958095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:45.958095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:45.959276 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:45.959276 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:45.959276 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:45.959276 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:45.959276 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:45.959276 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:45.959276 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:45.959276 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:45.959276 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:45.959276 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:45.959276 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:45.959276 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:13:45.959276 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 142/282] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:45.964230 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519c959-b7b5-40b0-93bc-e045e9ba3e1f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:45.975031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:46.022122 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0f947-f64a-4d20-964b-c58a308979ed None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:46.022410 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0f947-f64a-4d20-964b-c58a308979ed None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:46.022643 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0f947-f64a-4d20-964b-c58a308979ed None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:46.024185 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 143/283] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:46.030685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3de60-0bd9-4677-80ea-38544f3bbaee None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:46.031252 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3de60-0bd9-4677-80ea-38544f3bbaee None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:46.031717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de3de60-0bd9-4677-80ea-38544f3bbaee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:46.054500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2de3de60-0bd9-4677-80ea-38544f3bbaee None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:46.056517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2de3de60-0bd9-4677-80ea-38544f3bbaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:46.144465 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519c959-b7b5-40b0-93bc-e045e9ba3e1f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:46.160232 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 144/284] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 09:13:46.192300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10015ea-7766-432b-a656-7a72f19befdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:46.192971 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10015ea-7766-432b-a656-7a72f19befdc None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:46.193310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10015ea-7766-432b-a656-7a72f19befdc None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:46.193582 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10015ea-7766-432b-a656-7a72f19befdc None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:46.194296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10015ea-7766-432b-a656-7a72f19befdc None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:46.194600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10015ea-7766-432b-a656-7a72f19befdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:46.195980 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10015ea-7766-432b-a656-7a72f19befdc None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:46.207648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 141/285] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Sep 2 09:13:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:46.212767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3de60-0bd9-4677-80ea-38544f3bbaee None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:46.224967 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:46.315399 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de3de60-0bd9-4677-80ea-38544f3bbaee None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:46.358980 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5888998c-3562-41c1-975d-a6bb7b79ded1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:46.359731 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888998c-3562-41c1-975d-a6bb7b79ded1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:46.360015 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888998c-3562-41c1-975d-a6bb7b79ded1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:46.360287 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888998c-3562-41c1-975d-a6bb7b79ded1 None admin] PATH_INFO: `/v3/users/b28f835d8ca443768cc83703b41ae12b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:46.361470 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5888998c-3562-41c1-975d-a6bb7b79ded1 None admin] RBAC: Authorizing `identity:get_user(user_id=b28f835d8ca443768cc83703b41ae12b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:46.362940 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5888998c-3562-41c1-975d-a6bb7b79ded1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:46.364514 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 145/286] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:46 2022] GET /identity/v3/users/b28f835d8ca443768cc83703b41ae12b => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:46.370112 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10015ea-7766-432b-a656-7a72f19befdc None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:46.450189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:46.455027 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5888998c-3562-41c1-975d-a6bb7b79ded1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:46.486850 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10015ea-7766-432b-a656-7a72f19befdc None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:46.522063 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7086e9-24cc-4e0a-aef5-4eb73f577651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:46.522733 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7086e9-24cc-4e0a-aef5-4eb73f577651 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:46.523092 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7086e9-24cc-4e0a-aef5-4eb73f577651 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:46.523365 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7086e9-24cc-4e0a-aef5-4eb73f577651 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:46.619059 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:46.630145 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7086e9-24cc-4e0a-aef5-4eb73f577651 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:46.631681 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7086e9-24cc-4e0a-aef5-4eb73f577651 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:46.647367 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5888998c-3562-41c1-975d-a6bb7b79ded1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:46.735439 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a0c39d-084b-4b1d-915e-d5874152ca5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:46.736148 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0c39d-084b-4b1d-915e-d5874152ca5b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:46.736447 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0c39d-084b-4b1d-915e-d5874152ca5b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:46.736726 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0c39d-084b-4b1d-915e-d5874152ca5b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:46.737337 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:46.757389 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a0c39d-084b-4b1d-915e-d5874152ca5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:46.769014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-11a0c39d-084b-4b1d-915e-d5874152ca5b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 09:13:46.769014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:46.769014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:46.769014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:46.769014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:46.769014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:46.769014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:46.769014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:46.769014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:46.769014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:46.769014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:46.769014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:46.769014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:46.769014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:46.769014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:46.769014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:46.770374 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:46.770374 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:46.770374 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:46.770374 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:46.770374 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:46.770374 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:46.770374 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:46.770374 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:46.770374 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:46.770374 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:46.770374 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:46.770374 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:46.770374 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:46.770374 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:46.770374 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:46.770374 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:46.771190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:46.771190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:46.771190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:46.771190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:46.771190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:46.771190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:46.771190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:46.771190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:46.771190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:46.771190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:46.771190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 09:13:46.771190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:13:46.773863 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 146/287] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:13:46.779754 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a0c39d-084b-4b1d-915e-d5874152ca5b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:46.780962 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 142/288] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:13:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:13:46.786095 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7086e9-24cc-4e0a-aef5-4eb73f577651 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:46.935874 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7086e9-24cc-4e0a-aef5-4eb73f577651 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:46.976492 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865cfd6-169a-4d67-b0b4-0923d61691fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:46.977130 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865cfd6-169a-4d67-b0b4-0923d61691fb None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:46.977409 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865cfd6-169a-4d67-b0b4-0923d61691fb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:46.977686 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865cfd6-169a-4d67-b0b4-0923d61691fb None admin] PATH_INFO: `/v3/projects/dac3a1a96e914a549133e73f03943618` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:46.978852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3865cfd6-169a-4d67-b0b4-0923d61691fb None admin] RBAC: Authorizing `identity:get_project(project_id=dac3a1a96e914a549133e73f03943618)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:46.980461 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3865cfd6-169a-4d67-b0b4-0923d61691fb None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:46.982574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 143/289] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:46 2022] GET /identity/v3/projects/dac3a1a96e914a549133e73f03943618 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:13:46.988417 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865cfd6-169a-4d67-b0b4-0923d61691fb None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:47.024815 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a0c39d-084b-4b1d-915e-d5874152ca5b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:47.066022 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a32af-f1bc-4960-bb3b-b6c6163571e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:47.066636 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a32af-f1bc-4960-bb3b-b6c6163571e4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:47.066942 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a32af-f1bc-4960-bb3b-b6c6163571e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:47.067235 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a32af-f1bc-4960-bb3b-b6c6163571e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:47.068317 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a32af-f1bc-4960-bb3b-b6c6163571e4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:47.068629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a32af-f1bc-4960-bb3b-b6c6163571e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:47.070188 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a32af-f1bc-4960-bb3b-b6c6163571e4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:47.073501 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:47.073803 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:47.083075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 147/290] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Sep 2 09:13:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:13:47.091886 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a32af-f1bc-4960-bb3b-b6c6163571e4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:47.148893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865cfd6-169a-4d67-b0b4-0923d61691fb None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:47.201774 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a32af-f1bc-4960-bb3b-b6c6163571e4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:47.212954 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1e32d-a376-41b0-bfa7-0a9515d40960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:47.213554 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1e32d-a376-41b0-bfa7-0a9515d40960 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:47.213836 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1e32d-a376-41b0-bfa7-0a9515d40960 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:47.214146 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1e32d-a376-41b0-bfa7-0a9515d40960 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:47.214926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1e32d-a376-41b0-bfa7-0a9515d40960 None admin] RBAC: Adding query filter params (role.id=efe6bdd5112340b48b262ceafbc19c04, scope.project.id=dac3a1a96e914a549133e73f03943618, user.id=b28f835d8ca443768cc83703b41ae12b) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:47.215230 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1e32d-a376-41b0-bfa7-0a9515d40960 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:47.216660 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1e32d-a376-41b0-bfa7-0a9515d40960 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:47.223248 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 144/291] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:13:46 2022] GET /identity/v3/role_assignments?user.id=b28f835d8ca443768cc83703b41ae12b&scope.project.id=dac3a1a96e914a549133e73f03943618&role.id=efe6bdd5112340b48b262ceafbc19c04 => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:47.241816 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b8cda-fd56-4bf5-ad46-f43cff3cfbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:47.242924 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b8cda-fd56-4bf5-ad46-f43cff3cfbec None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:47.243412 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b8cda-fd56-4bf5-ad46-f43cff3cfbec None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:47.243890 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b8cda-fd56-4bf5-ad46-f43cff3cfbec None admin] PATH_INFO: `/v3/users/6845b0bb33f04b45b9ee70ce3b65db01` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:47.245940 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b8cda-fd56-4bf5-ad46-f43cff3cfbec None admin] RBAC: Authorizing `identity:get_user(user_id=6845b0bb33f04b45b9ee70ce3b65db01)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:47.247634 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b8cda-fd56-4bf5-ad46-f43cff3cfbec None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:47.249830 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 148/292] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:47 2022] GET /identity/v3/users/6845b0bb33f04b45b9ee70ce3b65db01 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:47.255277 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1e32d-a376-41b0-bfa7-0a9515d40960 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:47.390334 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:47.398014 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1e32d-a376-41b0-bfa7-0a9515d40960 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:47.433538 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794024d-e019-4b40-a467-7cde94139851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:47.434178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794024d-e019-4b40-a467-7cde94139851 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:47.434454 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794024d-e019-4b40-a467-7cde94139851 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:47.434747 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794024d-e019-4b40-a467-7cde94139851 None admin] PATH_INFO: `/v3/projects/dac3a1a96e914a549133e73f03943618` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:47.435859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794024d-e019-4b40-a467-7cde94139851 None admin] RBAC: Authorizing `identity:get_project(project_id=dac3a1a96e914a549133e73f03943618)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:47.437278 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794024d-e019-4b40-a467-7cde94139851 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:47.438918 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 145/293] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:47 2022] GET /identity/v3/projects/dac3a1a96e914a549133e73f03943618 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:47.444628 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b8cda-fd56-4bf5-ad46-f43cff3cfbec None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:47.520485 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:47.558674 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b8cda-fd56-4bf5-ad46-f43cff3cfbec None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:47.595701 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaceef0-cbbf-477c-aff2-0a668da3f651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:47.596429 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaceef0-cbbf-477c-aff2-0a668da3f651 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:47.596741 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaceef0-cbbf-477c-aff2-0a668da3f651 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:47.597016 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaceef0-cbbf-477c-aff2-0a668da3f651 None admin] PATH_INFO: `/v3/projects/dac3a1a96e914a549133e73f03943618/users/6845b0bb33f04b45b9ee70ce3b65db01/roles/b381d11646e54d1197676600202a6144` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:47.599014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaceef0-cbbf-477c-aff2-0a668da3f651 None admin] RBAC: Authorizing `identity:create_grant(project_id=dac3a1a96e914a549133e73f03943618, user_id=6845b0bb33f04b45b9ee70ce3b65db01, role_id=b381d11646e54d1197676600202a6144)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:47.600432 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaceef0-cbbf-477c-aff2-0a668da3f651 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:47.653542 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958383c-5ccc-4be4-95b3-761f1e82db69 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:47.653875 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958383c-5ccc-4be4-95b3-761f1e82db69 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:47.654142 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958383c-5ccc-4be4-95b3-761f1e82db69 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:47.655770 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 146/294] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:13:47.660710 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdcdef-729a-49ce-8c97-48973666eea8 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:47.660991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdcdef-729a-49ce-8c97-48973666eea8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:47.661260 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fdcdef-729a-49ce-8c97-48973666eea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:47.683029 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 149/295] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Sep 2 09:13:47 2022] PUT /identity/v3/projects/dac3a1a96e914a549133e73f03943618/users/6845b0bb33f04b45b9ee70ce3b65db01/roles/b381d11646e54d1197676600202a6144 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:13:47.691678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e5fdcdef-729a-49ce-8c97-48973666eea8 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:47.693393 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e5fdcdef-729a-49ce-8c97-48973666eea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:47.722819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e4a5c-bf08-4dc2-abe9-42674457f564 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:47.723140 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e4a5c-bf08-4dc2-abe9-42674457f564 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:47.723385 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e4a5c-bf08-4dc2-abe9-42674457f564 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:47.725021 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 150/296] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:47.730537 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ba705-2eeb-4b19-b1e1-608859920378 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:47.730877 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ba705-2eeb-4b19-b1e1-608859920378 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:47.731154 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990ba705-2eeb-4b19-b1e1-608859920378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:47.751081 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-990ba705-2eeb-4b19-b1e1-608859920378 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:47.752320 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-990ba705-2eeb-4b19-b1e1-608859920378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:47.782100 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 147/297] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:47.835971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 151/298] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 09:13:47.917657 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80381885-65de-4796-91d8-0998a1b847d7 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:47.918039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80381885-65de-4796-91d8-0998a1b847d7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:47.918390 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80381885-65de-4796-91d8-0998a1b847d7 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:47.919881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 148/299] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:47.925238 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce440f-a5fc-45c5-bffc-fdfc380151c8 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:47.925529 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce440f-a5fc-45c5-bffc-fdfc380151c8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:47.925806 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce440f-a5fc-45c5-bffc-fdfc380151c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:47.944497 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4cce440f-a5fc-45c5-bffc-fdfc380151c8 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:47.945693 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4cce440f-a5fc-45c5-bffc-fdfc380151c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:47.997818 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:48.015299 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 152/300] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:48.061764 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80381885-65de-4796-91d8-0998a1b847d7 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:48.122638 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce440f-a5fc-45c5-bffc-fdfc380151c8 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:48.176001 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:48.185801 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80381885-65de-4796-91d8-0998a1b847d7 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:48.226256 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce440f-a5fc-45c5-bffc-fdfc380151c8 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:48.232492 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f37f39-2014-47b0-859f-9a15c32b1821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:48.233133 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f37f39-2014-47b0-859f-9a15c32b1821 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:48.233413 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f37f39-2014-47b0-859f-9a15c32b1821 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:48.233687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f37f39-2014-47b0-859f-9a15c32b1821 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:48.248570 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f37f39-2014-47b0-859f-9a15c32b1821 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:48.255513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-74f37f39-2014-47b0-859f-9a15c32b1821 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:48.255513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:48.255513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:48.255513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:48.255513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:48.255513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:48.255513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:48.255513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:48.255513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:48.255513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:48.255513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:48.255513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:48.255513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:48.255513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:48.255513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:48.255513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:48.256378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:48.256378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:48.256378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:48.256378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:48.256378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:48.256378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:48.256378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:48.256378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:48.256378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:48.256378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:48.256378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:48.256378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:48.256378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:48.256378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:48.256378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:48.256378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:48.257222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:48.257222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:48.257222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:48.257222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:48.257222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:48.257222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:48.257222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:48.257222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:48.257222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:48.257222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:48.257222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:48.257222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:13:48.259745 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 149/301] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:48.265438 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f37f39-2014-47b0-859f-9a15c32b1821 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:48.272883 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228e70a-cfb3-4025-ab2c-8019c102727d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:48.273792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228e70a-cfb3-4025-ab2c-8019c102727d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:48.274270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228e70a-cfb3-4025-ab2c-8019c102727d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:48.274772 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228e70a-cfb3-4025-ab2c-8019c102727d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:48.276634 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228e70a-cfb3-4025-ab2c-8019c102727d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:48.278765 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228e70a-cfb3-4025-ab2c-8019c102727d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:48.281525 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 153/302] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:13:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:13:48.286819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228e70a-cfb3-4025-ab2c-8019c102727d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:48.353061 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:48.385539 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f37f39-2014-47b0-859f-9a15c32b1821 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:48.398352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228e70a-cfb3-4025-ab2c-8019c102727d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:48.423520 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe13d1a-992f-4ce4-8337-ca5293923c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:48.499670 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe13d1a-992f-4ce4-8337-ca5293923c53 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:48.499670 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe13d1a-992f-4ce4-8337-ca5293923c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:48.499670 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe13d1a-992f-4ce4-8337-ca5293923c53 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:48.499670 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe13d1a-992f-4ce4-8337-ca5293923c53 None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:48.499670 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe13d1a-992f-4ce4-8337-ca5293923c53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:48.499670 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe13d1a-992f-4ce4-8337-ca5293923c53 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:48.500124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8024029a-5402-4877-91cb-a2ea4d11bea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:48.500229 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 150/303] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Sep 2 09:13:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:13:48.500302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024029a-5402-4877-91cb-a2ea4d11bea1 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:48.500302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024029a-5402-4877-91cb-a2ea4d11bea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:48.500440 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe13d1a-992f-4ce4-8337-ca5293923c53 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:48.500519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8024029a-5402-4877-91cb-a2ea4d11bea1 None admin] PATH_INFO: `/v3/groups` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:48.500519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8024029a-5402-4877-91cb-a2ea4d11bea1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:48.500519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8024029a-5402-4877-91cb-a2ea4d11bea1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:48.500714 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:48.538694 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe13d1a-992f-4ce4-8337-ca5293923c53 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:48.540908 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 154/304] 10.222.0.57 () {60 vars in 1134 bytes} [Fri Sep 2 09:13:48 2022] POST /identity/v3/groups => generated 229 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:13:48.542494 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8024029a-5402-4877-91cb-a2ea4d11bea1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:48.595737 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a5d68-e251-4444-ba16-5213a5ea3567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:48.596558 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a5d68-e251-4444-ba16-5213a5ea3567 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:48.597911 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a5d68-e251-4444-ba16-5213a5ea3567 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:48.598246 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2a5d68-e251-4444-ba16-5213a5ea3567 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:48.616096 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2a5d68-e251-4444-ba16-5213a5ea3567 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:48.623783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-ab2a5d68-e251-4444-ba16-5213a5ea3567 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 09:13:48.623783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:48.623783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:48.623783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:48.623783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:48.623783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:48.623783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:48.623783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:48.623783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:48.623783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:48.623783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:48.623783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:48.623783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:48.623783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:48.623783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:48.623783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:48.624801 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:48.624801 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:48.624801 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:48.624801 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:48.624801 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:48.624801 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:48.624801 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:48.624801 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:48.624801 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:48.624801 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:48.624801 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:48.624801 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:48.624801 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:48.624801 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:48.624801 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:48.624801 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:48.625622 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:48.625622 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:48.625622 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:48.625622 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:48.625622 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:48.625622 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:48.625622 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:48.625622 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:48.625622 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:48.625622 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:48.625622 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 09:13:48.625622 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:13:48.625622 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 151/305] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:48.632854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a5d68-e251-4444-ba16-5213a5ea3567 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:48.682263 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8024029a-5402-4877-91cb-a2ea4d11bea1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:48.697598 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:48.722255 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556979c-9384-4473-b5a3-ccefe58eeb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:48.723101 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556979c-9384-4473-b5a3-ccefe58eeb9f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:48.723401 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556979c-9384-4473-b5a3-ccefe58eeb9f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:48.723664 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556979c-9384-4473-b5a3-ccefe58eeb9f None admin] PATH_INFO: `/v3/users/334c10c6eb4c4b1a9da1ed17c075a943` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:48.725285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556979c-9384-4473-b5a3-ccefe58eeb9f None admin] RBAC: Authorizing `identity:get_user(user_id=334c10c6eb4c4b1a9da1ed17c075a943)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:48.725756 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:48.727949 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8556979c-9384-4473-b5a3-ccefe58eeb9f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:48.730935 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 155/306] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:48 2022] GET /identity/v3/users/334c10c6eb4c4b1a9da1ed17c075a943 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:48.735941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556979c-9384-4473-b5a3-ccefe58eeb9f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:48.756289 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2a5d68-e251-4444-ba16-5213a5ea3567 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:48.800990 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d90bd-ec11-4cb1-a6a3-1446efa991eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:48.801608 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d90bd-ec11-4cb1-a6a3-1446efa991eb None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:48.801888 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d90bd-ec11-4cb1-a6a3-1446efa991eb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:48.802224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2d90bd-ec11-4cb1-a6a3-1446efa991eb None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:48.802934 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d90bd-ec11-4cb1-a6a3-1446efa991eb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:48.803240 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d90bd-ec11-4cb1-a6a3-1446efa991eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:48.804637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2d90bd-ec11-4cb1-a6a3-1446efa991eb None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:48.817723 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 152/307] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Sep 2 09:13:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:13:48.823596 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d90bd-ec11-4cb1-a6a3-1446efa991eb None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:48.883823 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8556979c-9384-4473-b5a3-ccefe58eeb9f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:48.919250 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a65199-2e47-48c6-a7d3-79a80fa9c270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:48.919903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a65199-2e47-48c6-a7d3-79a80fa9c270 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:48.920187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a65199-2e47-48c6-a7d3-79a80fa9c270 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:48.920459 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a65199-2e47-48c6-a7d3-79a80fa9c270 None admin] PATH_INFO: `/v3/system/users/334c10c6eb4c4b1a9da1ed17c075a943/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:48.921869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a65199-2e47-48c6-a7d3-79a80fa9c270 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=334c10c6eb4c4b1a9da1ed17c075a943, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:48.923611 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a65199-2e47-48c6-a7d3-79a80fa9c270 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:48.949108 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2d90bd-ec11-4cb1-a6a3-1446efa991eb None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:48.985485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece91b6-2bf9-417b-a001-971383a70012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:48.986121 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece91b6-2bf9-417b-a001-971383a70012 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:48.986459 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece91b6-2bf9-417b-a001-971383a70012 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:48.986768 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece91b6-2bf9-417b-a001-971383a70012 None admin] PATH_INFO: `/v3/users/447c8e27c7f34d5493aad67ad5d60f11` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:48.997979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:49.001693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aece91b6-2bf9-417b-a001-971383a70012 None admin] RBAC: Authorizing `identity:get_user(user_id=447c8e27c7f34d5493aad67ad5d60f11)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:49.003375 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aece91b6-2bf9-417b-a001-971383a70012 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:49.010739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 153/309] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:48 2022] GET /identity/v3/users/447c8e27c7f34d5493aad67ad5d60f11 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:13:49.011293 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 156/309] 10.222.0.57 () {58 vars in 1334 bytes} [Fri Sep 2 09:13:48 2022] PUT /identity/v3/system/users/334c10c6eb4c4b1a9da1ed17c075a943/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:13:49.023691 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece91b6-2bf9-417b-a001-971383a70012 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:49.122900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece91b6-2bf9-417b-a001-971383a70012 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:49.166646 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550053f-db19-42ec-a2d9-2e02a6f4be64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:49.167656 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550053f-db19-42ec-a2d9-2e02a6f4be64 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:49.168168 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550053f-db19-42ec-a2d9-2e02a6f4be64 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:49.168651 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550053f-db19-42ec-a2d9-2e02a6f4be64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:49.169593 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e550053f-db19-42ec-a2d9-2e02a6f4be64 None admin] RBAC: Adding query filter params (role.id=b381d11646e54d1197676600202a6144, user.id=447c8e27c7f34d5493aad67ad5d60f11, scope.system=all) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:49.170117 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e550053f-db19-42ec-a2d9-2e02a6f4be64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:49.171863 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e550053f-db19-42ec-a2d9-2e02a6f4be64 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:49.178329 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 154/310] 10.222.0.57 () {56 vars in 1308 bytes} [Fri Sep 2 09:13:49 2022] GET /identity/v3/role_assignments?user.id=447c8e27c7f34d5493aad67ad5d60f11&scope.system=all&role.id=b381d11646e54d1197676600202a6144 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:49.681137 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3452b-5002-441f-a363-2a9cbdbdc2c0 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:49.681588 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3452b-5002-441f-a363-2a9cbdbdc2c0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:49.682530 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda3452b-5002-441f-a363-2a9cbdbdc2c0 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:49.684411 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 157/311] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:49.690030 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdddd5-5552-4349-a61d-9377995dd600 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:49.690326 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdddd5-5552-4349-a61d-9377995dd600 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:49.690604 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdddd5-5552-4349-a61d-9377995dd600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:49.710242 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-decdddd5-5552-4349-a61d-9377995dd600 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:49.711510 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-decdddd5-5552-4349-a61d-9377995dd600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:49.770655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:49.784619 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 155/312] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:50.063652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda3452b-5002-441f-a363-2a9cbdbdc2c0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:50.115616 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:50.175137 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda3452b-5002-441f-a363-2a9cbdbdc2c0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:50.212051 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afeaff5-c1da-46eb-b16b-53a958e73a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:50.213485 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afeaff5-c1da-46eb-b16b-53a958e73a35 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:50.213912 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afeaff5-c1da-46eb-b16b-53a958e73a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:50.214347 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afeaff5-c1da-46eb-b16b-53a958e73a35 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:50.224891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afeaff5-c1da-46eb-b16b-53a958e73a35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:50.230944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-0afeaff5-c1da-46eb-b16b-53a958e73a35 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 09:13:50.230944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:50.230944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:50.230944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:50.230944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:50.230944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:50.230944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:50.230944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:50.230944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:50.230944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:50.230944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:50.230944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:50.230944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:50.230944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:50.230944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:50.230944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:50.232089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:50.232089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:50.232089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:50.232089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:50.232089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:50.232089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:50.232089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:50.232089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:50.232089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:50.232089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:50.232089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:50.232089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:50.232089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:50.232089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:50.232089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:50.232089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:50.233089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:50.233089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:50.233089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:50.233089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:50.233089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:50.233089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:50.233089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:50.233089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:50.233089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:50.233089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:50.233089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 09:13:50.233089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:13:50.240197 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 158/313] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:50.241732 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decdddd5-5552-4349-a61d-9377995dd600 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:50.345671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decdddd5-5552-4349-a61d-9377995dd600 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:50.380309 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f5d2c-2f86-4302-93b7-3ba6ebcd5d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:50.381076 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f5d2c-2f86-4302-93b7-3ba6ebcd5d9a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:50.381455 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f5d2c-2f86-4302-93b7-3ba6ebcd5d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:50.381827 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f5d2c-2f86-4302-93b7-3ba6ebcd5d9a None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:50.382600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f5d2c-2f86-4302-93b7-3ba6ebcd5d9a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:50.383021 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f5d2c-2f86-4302-93b7-3ba6ebcd5d9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:50.384482 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f5d2c-2f86-4302-93b7-3ba6ebcd5d9a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:50.393316 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 156/314] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Sep 2 09:13:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:50.399763 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afeaff5-c1da-46eb-b16b-53a958e73a35 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:50.500760 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afeaff5-c1da-46eb-b16b-53a958e73a35 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:50.507827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:50.536084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fbf9ee-579e-4397-a1b9-a609c5986eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:50.536717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbf9ee-579e-4397-a1b9-a609c5986eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:50.536990 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbf9ee-579e-4397-a1b9-a609c5986eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:50.537262 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbf9ee-579e-4397-a1b9-a609c5986eb0 None admin] PATH_INFO: `/v3/users/6845b0bb33f04b45b9ee70ce3b65db01` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:50.538391 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fbf9ee-579e-4397-a1b9-a609c5986eb0 None admin] RBAC: Authorizing `identity:get_user(user_id=6845b0bb33f04b45b9ee70ce3b65db01)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:50.539860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fbf9ee-579e-4397-a1b9-a609c5986eb0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:50.543823 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 159/315] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:50 2022] GET /identity/v3/users/6845b0bb33f04b45b9ee70ce3b65db01 => generated 297 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:50.549379 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f5d2c-2f86-4302-93b7-3ba6ebcd5d9a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:50.608428 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:50.616791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45784aa0-8faa-4839-840c-2f135ed7aafb None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:50.624497 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45784aa0-8faa-4839-840c-2f135ed7aafb None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:50.624497 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45784aa0-8faa-4839-840c-2f135ed7aafb None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:50.624497 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 160/316] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:50.624497 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a39b043-4695-49c3-ac76-247beb981e9e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:50.624497 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a39b043-4695-49c3-ac76-247beb981e9e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:50.624497 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a39b043-4695-49c3-ac76-247beb981e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:50.643227 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9a39b043-4695-49c3-ac76-247beb981e9e None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:50.647461 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9a39b043-4695-49c3-ac76-247beb981e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:50.661989 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f5d2c-2f86-4302-93b7-3ba6ebcd5d9a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:50.710160 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78390ffe-3739-47e0-8213-4ef3b5e22d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:50.710803 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78390ffe-3739-47e0-8213-4ef3b5e22d83 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:50.711080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78390ffe-3739-47e0-8213-4ef3b5e22d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:50.711385 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78390ffe-3739-47e0-8213-4ef3b5e22d83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:50.728108 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 161/317] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 09:13:50.788267 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78390ffe-3739-47e0-8213-4ef3b5e22d83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:50.790143 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78390ffe-3739-47e0-8213-4ef3b5e22d83 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:50.850741 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:50.895703 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 157/318] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:13:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:13:50.901657 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a39b043-4695-49c3-ac76-247beb981e9e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:50.975787 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58727a26-57a5-452e-a968-333e3ca37a02 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:50.976077 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58727a26-57a5-452e-a968-333e3ca37a02 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:50.976333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58727a26-57a5-452e-a968-333e3ca37a02 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:50.977693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 158/319] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:50.983390 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f955851-2221-4d18-8a0c-a88c3d43c15f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:50.983543 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f955851-2221-4d18-8a0c-a88c3d43c15f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:50.983741 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f955851-2221-4d18-8a0c-a88c3d43c15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:51.001602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-9f955851-2221-4d18-8a0c-a88c3d43c15f None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:51.002930 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9f955851-2221-4d18-8a0c-a88c3d43c15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:51.014826 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a39b043-4695-49c3-ac76-247beb981e9e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:51.017913 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:51.052795 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8286787-0cb1-44fd-8bf5-73ee9548a658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:51.053695 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8286787-0cb1-44fd-8bf5-73ee9548a658 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:51.053979 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8286787-0cb1-44fd-8bf5-73ee9548a658 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:51.054251 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8286787-0cb1-44fd-8bf5-73ee9548a658 None admin] PATH_INFO: `/v3/projects/dac3a1a96e914a549133e73f03943618` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:51.055971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8286787-0cb1-44fd-8bf5-73ee9548a658 None admin] RBAC: Authorizing `identity:get_project(project_id=dac3a1a96e914a549133e73f03943618)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:51.057447 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8286787-0cb1-44fd-8bf5-73ee9548a658 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:51.064477 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 162/320] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:50 2022] GET /identity/v3/projects/dac3a1a96e914a549133e73f03943618 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:51.064477 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8286787-0cb1-44fd-8bf5-73ee9548a658 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:51.071692 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 159/321] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:51.072397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f955851-2221-4d18-8a0c-a88c3d43c15f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:51.167413 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:51.185537 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f955851-2221-4d18-8a0c-a88c3d43c15f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:51.189380 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8286787-0cb1-44fd-8bf5-73ee9548a658 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:51.223190 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dce1ba-666d-4365-8953-b234b93e2273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:51.223788 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dce1ba-666d-4365-8953-b234b93e2273 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:51.224065 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dce1ba-666d-4365-8953-b234b93e2273 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:51.224333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dce1ba-666d-4365-8953-b234b93e2273 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:51.225046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dce1ba-666d-4365-8953-b234b93e2273 None admin] RBAC: Adding query filter params (role.id=b381d11646e54d1197676600202a6144, scope.project.id=dac3a1a96e914a549133e73f03943618, user.id=6845b0bb33f04b45b9ee70ce3b65db01) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:51.225145 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09063b79-fb3f-4c41-af1f-d0bff66380fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:51.225339 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dce1ba-666d-4365-8953-b234b93e2273 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:51.225740 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09063b79-fb3f-4c41-af1f-d0bff66380fc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:51.226019 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09063b79-fb3f-4c41-af1f-d0bff66380fc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:51.226288 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09063b79-fb3f-4c41-af1f-d0bff66380fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:51.226834 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dce1ba-666d-4365-8953-b234b93e2273 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:51.227418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09063b79-fb3f-4c41-af1f-d0bff66380fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:51.228851 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09063b79-fb3f-4c41-af1f-d0bff66380fc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:51.230628 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 163/322] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:13:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:51.238146 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 160/323] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:13:51 2022] GET /identity/v3/role_assignments?user.id=6845b0bb33f04b45b9ee70ce3b65db01&scope.project.id=dac3a1a96e914a549133e73f03943618&role.id=b381d11646e54d1197676600202a6144 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:51.238307 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6330e-f1ce-4460-a7b4-ef54844be241 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:51.238307 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6330e-f1ce-4460-a7b4-ef54844be241 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:51.238307 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6330e-f1ce-4460-a7b4-ef54844be241 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:51.238307 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 164/324] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:51.242822 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dce1ba-666d-4365-8953-b234b93e2273 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:51.244490 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d2657-f99f-4e72-9cd3-1aa470a4b20b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:51.244771 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d2657-f99f-4e72-9cd3-1aa470a4b20b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:51.245037 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d2657-f99f-4e72-9cd3-1aa470a4b20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:51.266380 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4e2d2657-f99f-4e72-9cd3-1aa470a4b20b None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:51.267720 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4e2d2657-f99f-4e72-9cd3-1aa470a4b20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:51.317152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:51.354863 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 165/325] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:51.355903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d2657-f99f-4e72-9cd3-1aa470a4b20b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:51.370292 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dce1ba-666d-4365-8953-b234b93e2273 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:51.410461 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669aee7-af77-4963-9f42-a78c9e9b94ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:51.411280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669aee7-af77-4963-9f42-a78c9e9b94ab None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:51.411561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669aee7-af77-4963-9f42-a78c9e9b94ab None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:51.411834 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669aee7-af77-4963-9f42-a78c9e9b94ab None admin] PATH_INFO: `/v3/groups` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:51.412502 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f669aee7-af77-4963-9f42-a78c9e9b94ab None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:51.414546 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f669aee7-af77-4963-9f42-a78c9e9b94ab None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:51.475970 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d2657-f99f-4e72-9cd3-1aa470a4b20b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:51.512399 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0cbd9-a93e-417a-beed-eecd59d011cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:51.513032 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0cbd9-a93e-417a-beed-eecd59d011cc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:51.515844 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0cbd9-a93e-417a-beed-eecd59d011cc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:51.515844 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0cbd9-a93e-417a-beed-eecd59d011cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:51.524009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0cbd9-a93e-417a-beed-eecd59d011cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:51.528858 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-7af0cbd9-a93e-417a-beed-eecd59d011cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:51.528858 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:51.528858 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:51.528858 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:51.528858 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:51.528858 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:51.528858 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:51.528858 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:51.528858 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:51.528858 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:51.528858 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:51.528858 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:51.528858 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:51.528858 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:51.528858 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:51.528858 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:51.529805 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:51.529805 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:51.529805 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:51.529805 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:51.529805 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:51.529805 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:51.529805 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:51.529805 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:51.529805 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:51.529805 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:51.529805 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:51.529805 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:51.529805 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:51.529805 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:51.529805 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:51.529805 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:51.530605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:51.530605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:51.530605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:51.530605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:51.530605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:51.530605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:51.530605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:51.530605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:51.530605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:51.530605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:51.530605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:51.530605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:13:51.530605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 166/326] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:51.535636 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0cbd9-a93e-417a-beed-eecd59d011cc None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:51.541314 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 161/327] 10.222.0.57 () {60 vars in 1134 bytes} [Fri Sep 2 09:13:51 2022] POST /identity/v3/groups => generated 226 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:13:51.570421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:51.630728 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669aee7-af77-4963-9f42-a78c9e9b94ab None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:51.645220 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:51.651867 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0cbd9-a93e-417a-beed-eecd59d011cc None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:51.692262 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5d339-d502-4cbf-a899-ff1338292b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:51.692867 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5d339-d502-4cbf-a899-ff1338292b3c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:51.693146 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5d339-d502-4cbf-a899-ff1338292b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:51.693683 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5d339-d502-4cbf-a899-ff1338292b3c None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:51.694085 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd5d339-d502-4cbf-a899-ff1338292b3c None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:51.694388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd5d339-d502-4cbf-a899-ff1338292b3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:51.695777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd5d339-d502-4cbf-a899-ff1338292b3c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:51.706399 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 167/328] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Sep 2 09:13:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:13:51.713371 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5d339-d502-4cbf-a899-ff1338292b3c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:51.750154 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669aee7-af77-4963-9f42-a78c9e9b94ab None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:51.794373 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ecfd0d-a8db-472c-a9e0-1d05d136c005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:51.795342 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ecfd0d-a8db-472c-a9e0-1d05d136c005 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:51.795798 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ecfd0d-a8db-472c-a9e0-1d05d136c005 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:51.796226 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ecfd0d-a8db-472c-a9e0-1d05d136c005 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:51.808181 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ecfd0d-a8db-472c-a9e0-1d05d136c005 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:51.813773 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-35ecfd0d-a8db-472c-a9e0-1d05d136c005 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 09:13:51.813773 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:51.813773 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:51.813773 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:51.813773 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:51.813773 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:51.813773 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:51.813773 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:51.813773 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:51.813773 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:51.813773 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:51.813773 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:51.813773 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:51.813773 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:51.813773 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:51.813773 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:51.814896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:51.814896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:51.814896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:51.814896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:51.814896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:51.814896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:51.814896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:51.814896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:51.814896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:51.814896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:51.814896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:51.814896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:51.814896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:51.814896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:51.814896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:51.814896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:51.815894 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:51.815894 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:51.815894 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:51.815894 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:51.815894 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:51.815894 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:51.815894 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:51.815894 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:51.815894 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:51.815894 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:51.815894 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 09:13:51.815894 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:13:51.817831 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 162/329] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:13:51.823800 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ecfd0d-a8db-472c-a9e0-1d05d136c005 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:51.834814 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:51.848453 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5d339-d502-4cbf-a899-ff1338292b3c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:51.892169 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba1b95-0d0f-4c78-a587-41de20170126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:51.892915 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba1b95-0d0f-4c78-a587-41de20170126 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:51.893256 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba1b95-0d0f-4c78-a587-41de20170126 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:51.893613 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba1b95-0d0f-4c78-a587-41de20170126 None admin] PATH_INFO: `/v3/users/334c10c6eb4c4b1a9da1ed17c075a943` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:51.894863 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba1b95-0d0f-4c78-a587-41de20170126 None admin] RBAC: Authorizing `identity:get_user(user_id=334c10c6eb4c4b1a9da1ed17c075a943)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:51.896383 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba1b95-0d0f-4c78-a587-41de20170126 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:51.898347 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 168/330] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:51 2022] GET /identity/v3/users/334c10c6eb4c4b1a9da1ed17c075a943 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:51.904318 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba1b95-0d0f-4c78-a587-41de20170126 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:51.913617 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:51.953075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ecfd0d-a8db-472c-a9e0-1d05d136c005 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:51.997131 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57e687-392c-4661-bfee-c21d495b1029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:51.997769 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57e687-392c-4661-bfee-c21d495b1029 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:51.998037 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57e687-392c-4661-bfee-c21d495b1029 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:51.998310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57e687-392c-4661-bfee-c21d495b1029 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:51.999015 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57e687-392c-4661-bfee-c21d495b1029 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:51.999323 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57e687-392c-4661-bfee-c21d495b1029 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:52.000749 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57e687-392c-4661-bfee-c21d495b1029 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:52.010349 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 163/331] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Sep 2 09:13:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:52.016044 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57e687-392c-4661-bfee-c21d495b1029 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:52.035259 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba1b95-0d0f-4c78-a587-41de20170126 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:52.077301 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa9cd9-b0ce-4678-af06-501c27011e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:52.078585 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa9cd9-b0ce-4678-af06-501c27011e16 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:52.079065 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa9cd9-b0ce-4678-af06-501c27011e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:52.079475 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa9cd9-b0ce-4678-af06-501c27011e16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:52.080363 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa9cd9-b0ce-4678-af06-501c27011e16 None admin] RBAC: Adding query filter params (role.id=efe6bdd5112340b48b262ceafbc19c04, user.id=334c10c6eb4c4b1a9da1ed17c075a943, scope.system=all) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:52.080823 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa9cd9-b0ce-4678-af06-501c27011e16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:52.082891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffa9cd9-b0ce-4678-af06-501c27011e16 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:52.097007 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 169/332] 10.222.0.57 () {56 vars in 1308 bytes} [Fri Sep 2 09:13:51 2022] GET /identity/v3/role_assignments?user.id=334c10c6eb4c4b1a9da1ed17c075a943&scope.system=all&role.id=efe6bdd5112340b48b262ceafbc19c04 => generated 510 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:13:52.154810 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57e687-392c-4661-bfee-c21d495b1029 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:52.188443 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:52.194576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9622d6-8c19-47ed-a7b5-5cfe7cc43541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:52.195302 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9622d6-8c19-47ed-a7b5-5cfe7cc43541 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:52.195585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9622d6-8c19-47ed-a7b5-5cfe7cc43541 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:52.195915 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9622d6-8c19-47ed-a7b5-5cfe7cc43541 None admin] PATH_INFO: `/v3/users/447c8e27c7f34d5493aad67ad5d60f11` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:52.197734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9622d6-8c19-47ed-a7b5-5cfe7cc43541 None admin] RBAC: Authorizing `identity:get_user(user_id=447c8e27c7f34d5493aad67ad5d60f11)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:52.199226 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9622d6-8c19-47ed-a7b5-5cfe7cc43541 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:52.201408 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 164/333] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:52 2022] GET /identity/v3/users/447c8e27c7f34d5493aad67ad5d60f11 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:52.207365 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa9cd9-b0ce-4678-af06-501c27011e16 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:52.234493 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:52.327217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa9cd9-b0ce-4678-af06-501c27011e16 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:52.365060 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ada66-dc23-46ed-ba76-a13dd72fe1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:52.365954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ada66-dc23-46ed-ba76-a13dd72fe1cc None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:52.366396 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ada66-dc23-46ed-ba76-a13dd72fe1cc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:52.366851 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ada66-dc23-46ed-ba76-a13dd72fe1cc None admin] PATH_INFO: `/v3/system/users/447c8e27c7f34d5493aad67ad5d60f11/roles/b381d11646e54d1197676600202a6144` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:52.368456 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ada66-dc23-46ed-ba76-a13dd72fe1cc None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=447c8e27c7f34d5493aad67ad5d60f11, role_id=b381d11646e54d1197676600202a6144)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:52.370560 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ada66-dc23-46ed-ba76-a13dd72fe1cc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:52.467068 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 170/334] 10.222.0.57 () {58 vars in 1334 bytes} [Fri Sep 2 09:13:52 2022] PUT /identity/v3/system/users/447c8e27c7f34d5493aad67ad5d60f11/roles/b381d11646e54d1197676600202a6144 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:13:53.765021 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0512d-8290-42e3-b62f-8eb50dff8250 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:53.766072 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0512d-8290-42e3-b62f-8eb50dff8250 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:53.766557 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0512d-8290-42e3-b62f-8eb50dff8250 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:53.768139 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 165/335] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:53.780257 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c506d-627e-48ff-9e2b-89ef20a818e7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:53.781141 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c506d-627e-48ff-9e2b-89ef20a818e7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:53.781628 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c506d-627e-48ff-9e2b-89ef20a818e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:53.813645 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d15c506d-627e-48ff-9e2b-89ef20a818e7 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:53.815212 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d15c506d-627e-48ff-9e2b-89ef20a818e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:53.910322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 171/336] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 02 09:13:54.063786 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b3032-8d66-41f0-b11a-158667f65689 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:54.064404 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b3032-8d66-41f0-b11a-158667f65689 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:54.064890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08b3032-8d66-41f0-b11a-158667f65689 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:54.066562 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 166/337] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:54.088649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609aa10-8fcd-48ea-9e3d-ee3ae6d3d3fe None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:54.089254 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609aa10-8fcd-48ea-9e3d-ee3ae6d3d3fe None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:54.089724 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609aa10-8fcd-48ea-9e3d-ee3ae6d3d3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:54.111219 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a609aa10-8fcd-48ea-9e3d-ee3ae6d3d3fe None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:54.112729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a609aa10-8fcd-48ea-9e3d-ee3ae6d3d3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:54.125207 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:54.193039 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 172/338] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:54.216518 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b3032-8d66-41f0-b11a-158667f65689 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:54.357157 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08b3032-8d66-41f0-b11a-158667f65689 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:54.396241 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc0cf0-daae-4fe6-837b-7c0e44441317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:54.396884 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc0cf0-daae-4fe6-837b-7c0e44441317 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:54.397163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc0cf0-daae-4fe6-837b-7c0e44441317 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:54.397439 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc0cf0-daae-4fe6-837b-7c0e44441317 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:54.411041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afc0cf0-daae-4fe6-837b-7c0e44441317 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:54.416424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-3afc0cf0-daae-4fe6-837b-7c0e44441317 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:54.416424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:54.416424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:54.416424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:54.416424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:54.416424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:54.416424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:54.416424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:54.416424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:54.416424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:54.416424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:54.416424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:54.416424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:54.416424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:54.416424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:54.416424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:54.417296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:54.417296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:54.417296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:54.417296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:54.417296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:54.417296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:54.417296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:54.417296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:54.417296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:54.417296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:54.417296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:54.417296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:54.417296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:54.417296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:54.417296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:54.417296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:54.418460 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:54.418460 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:54.418460 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:54.418460 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:54.418460 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:54.418460 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:54.418460 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:54.418460 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:54.418460 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:54.418460 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:54.418460 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:54.418460 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:13:54.418460 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 167/339] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:54.423729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609aa10-8fcd-48ea-9e3d-ee3ae6d3d3fe None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:54.537509 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609aa10-8fcd-48ea-9e3d-ee3ae6d3d3fe None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:54.579436 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc0cf0-daae-4fe6-837b-7c0e44441317 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:54.580582 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e5a3d-82ce-4f01-85ca-0d3966441466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:54.581216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e5a3d-82ce-4f01-85ca-0d3966441466 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:54.581532 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e5a3d-82ce-4f01-85ca-0d3966441466 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:54.581759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e5a3d-82ce-4f01-85ca-0d3966441466 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:54.582447 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e5a3d-82ce-4f01-85ca-0d3966441466 None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:54.582774 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e5a3d-82ce-4f01-85ca-0d3966441466 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:54.584155 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e5a3d-82ce-4f01-85ca-0d3966441466 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:54.598767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 173/340] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Sep 2 09:13:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:54.606310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e5a3d-82ce-4f01-85ca-0d3966441466 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:54.611046 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:54.664896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:54.744657 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc0cf0-daae-4fe6-837b-7c0e44441317 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:54.818598 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e5a3d-82ce-4f01-85ca-0d3966441466 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:54.850738 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a04273-5f49-419e-b494-e52847c08a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:54.851393 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a04273-5f49-419e-b494-e52847c08a2a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:54.851719 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a04273-5f49-419e-b494-e52847c08a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:54.851997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a04273-5f49-419e-b494-e52847c08a2a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:54.858520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88657da1-929c-4e6f-8d3a-5229ce108637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:54.859170 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88657da1-929c-4e6f-8d3a-5229ce108637 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:54.859449 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88657da1-929c-4e6f-8d3a-5229ce108637 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:54.859719 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88657da1-929c-4e6f-8d3a-5229ce108637 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:54.881343 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a04273-5f49-419e-b494-e52847c08a2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:54.887261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-96a04273-5f49-419e-b494-e52847c08a2a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 09:13:54.887261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:54.887261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:54.887261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:54.887261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:54.887261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:54.887261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:54.887261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:54.887261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:54.887261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:54.887261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:54.887261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:54.887261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:54.887261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:54.887261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:54.887261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:54.894082 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:54.894082 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:54.894082 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:54.894082 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:54.894082 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:54.894082 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:54.894082 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:54.894082 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:54.894082 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:54.894082 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:54.894082 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:54.894082 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:54.894082 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:54.894082 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:54.894082 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:54.894082 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:54.899612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:54.899612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:54.899612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:54.899612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:54.899612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:54.899612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:54.899612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:54.899612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:54.899612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:54.899612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:54.899612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 09:13:54.899612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:13:54.899612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 168/341] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Sep 2 09:13:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:13:54.899612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f53a0b-3e41-4514-bd33-4f98006d64c8 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:54.902101 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f53a0b-3e41-4514-bd33-4f98006d64c8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:54.902101 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f53a0b-3e41-4514-bd33-4f98006d64c8 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:54.902101 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 169/342] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:54 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:54.902101 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f53a0b-3e41-4514-bd33-4f98006d64c8 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:54.985347 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88657da1-929c-4e6f-8d3a-5229ce108637 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:54.986960 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88657da1-929c-4e6f-8d3a-5229ce108637 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:54.993014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:55.117679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f53a0b-3e41-4514-bd33-4f98006d64c8 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:55.153828 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:55.167878 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 174/343] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:13:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 02 09:13:55.169431 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fb190-9649-4e77-9bb4-322a48456cff None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:55.169738 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fb190-9649-4e77-9bb4-322a48456cff None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:55.170946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312fb190-9649-4e77-9bb4-322a48456cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:55.185955 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c7a8c-2341-417f-b4f1-a339b62a134d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:55.186587 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c7a8c-2341-417f-b4f1-a339b62a134d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:55.186890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c7a8c-2341-417f-b4f1-a339b62a134d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:55.187159 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c7a8c-2341-417f-b4f1-a339b62a134d None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:55.187825 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c7a8c-2341-417f-b4f1-a339b62a134d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:55.188128 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c7a8c-2341-417f-b4f1-a339b62a134d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:55.189498 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c7a8c-2341-417f-b4f1-a339b62a134d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:55.192124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-312fb190-9649-4e77-9bb4-322a48456cff None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:55.193327 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-312fb190-9649-4e77-9bb4-322a48456cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:55.216529 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 170/344] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:13:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:55.217500 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c7a8c-2341-417f-b4f1-a339b62a134d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:55.285220 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 175/345] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:55.286100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312fb190-9649-4e77-9bb4-322a48456cff None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:55.378815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c7a8c-2341-417f-b4f1-a339b62a134d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:55.379109 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:55.422149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a367b5-1eb2-440a-b489-6150f011720a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:55.422735 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a367b5-1eb2-440a-b489-6150f011720a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:55.424557 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a367b5-1eb2-440a-b489-6150f011720a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:55.424837 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a367b5-1eb2-440a-b489-6150f011720a None admin] PATH_INFO: `/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:55.426782 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a367b5-1eb2-440a-b489-6150f011720a None admin] RBAC: Authorizing `identity:get_project(project_id=79a2aa8010c54ad7ad7b27f08f4de74c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:55.431532 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a367b5-1eb2-440a-b489-6150f011720a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:55.433461 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 171/346] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:55 2022] GET /identity/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:55.440134 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a367b5-1eb2-440a-b489-6150f011720a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:55.540263 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312fb190-9649-4e77-9bb4-322a48456cff None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:55.562198 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:55.612976 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256682a-baf8-4285-80e2-e98c8810aa0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:55.613743 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256682a-baf8-4285-80e2-e98c8810aa0f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:55.613861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256682a-baf8-4285-80e2-e98c8810aa0f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:55.614123 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256682a-baf8-4285-80e2-e98c8810aa0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:55.663163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a367b5-1eb2-440a-b489-6150f011720a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:55.701436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c256682a-baf8-4285-80e2-e98c8810aa0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:55.702859 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c256682a-baf8-4285-80e2-e98c8810aa0f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:55.717245 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d6957-a28c-4f19-bed4-3d477e853341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:55.718239 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d6957-a28c-4f19-bed4-3d477e853341 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:55.718751 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d6957-a28c-4f19-bed4-3d477e853341 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:55.719238 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d6957-a28c-4f19-bed4-3d477e853341 None admin] PATH_INFO: `/v3/groups/8a53ec1153c1430ea8a4ac281c27da8b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:55.731330 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d6957-a28c-4f19-bed4-3d477e853341 None admin] RBAC: Authorizing `identity:get_group(group_id=8a53ec1153c1430ea8a4ac281c27da8b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:55.733137 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d6957-a28c-4f19-bed4-3d477e853341 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:55.735529 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 172/347] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Sep 2 09:13:55 2022] GET /identity/v3/groups/8a53ec1153c1430ea8a4ac281c27da8b => generated 226 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:55.739010 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:55.741638 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d6957-a28c-4f19-bed4-3d477e853341 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:55.810113 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 176/348] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:13:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:13:55.811077 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256682a-baf8-4285-80e2-e98c8810aa0f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:55.898108 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d6957-a28c-4f19-bed4-3d477e853341 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:55.912650 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:55.940597 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922eb53-8cf3-4abd-8168-5d48ddfa5846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:55.941197 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922eb53-8cf3-4abd-8168-5d48ddfa5846 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:55.941475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922eb53-8cf3-4abd-8168-5d48ddfa5846 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:55.941718 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256682a-baf8-4285-80e2-e98c8810aa0f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:55.942172 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922eb53-8cf3-4abd-8168-5d48ddfa5846 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:55.942461 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b922eb53-8cf3-4abd-8168-5d48ddfa5846 None admin] RBAC: Adding query filter params (group.id=8a53ec1153c1430ea8a4ac281c27da8b, role.id=efe6bdd5112340b48b262ceafbc19c04, scope.project.id=79a2aa8010c54ad7ad7b27f08f4de74c) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:55.943366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b922eb53-8cf3-4abd-8168-5d48ddfa5846 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:55.945436 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b922eb53-8cf3-4abd-8168-5d48ddfa5846 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:55.954931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 173/349] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Sep 2 09:13:55 2022] GET /identity/v3/role_assignments?group.id=8a53ec1153c1430ea8a4ac281c27da8b&scope.project.id=79a2aa8010c54ad7ad7b27f08f4de74c&role.id=efe6bdd5112340b48b262ceafbc19c04 => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:13:55.956922 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922eb53-8cf3-4abd-8168-5d48ddfa5846 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:55.983389 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5209b-f033-4126-b0e7-5ef80bcc3592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:55.984037 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5209b-f033-4126-b0e7-5ef80bcc3592 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:55.984312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5209b-f033-4126-b0e7-5ef80bcc3592 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:55.984579 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5209b-f033-4126-b0e7-5ef80bcc3592 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:55.987986 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:55.998836 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb5209b-f033-4126-b0e7-5ef80bcc3592 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:56.003841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-dbb5209b-f033-4126-b0e7-5ef80bcc3592 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:56.003841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:56.003841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:56.003841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:56.003841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:56.003841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:56.003841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:56.003841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:56.003841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:56.003841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:56.003841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:56.003841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:56.003841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:56.003841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:56.003841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:56.003841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:56.004799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:56.004799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:56.004799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:56.004799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:56.004799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:56.004799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:56.004799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:56.004799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:56.004799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:56.004799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:56.004799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:56.004799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:56.004799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:56.004799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:56.004799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:56.004799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:56.005649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:56.005649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:56.005649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:56.005649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:56.005649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:56.005649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:56.005649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:56.005649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:56.005649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:56.005649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:56.005649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:56.005649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:13:56.005649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 177/350] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:56.006950 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e347681-b93b-4e6b-b720-e2cba3599188 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:56.007235 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e347681-b93b-4e6b-b720-e2cba3599188 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:56.007502 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e347681-b93b-4e6b-b720-e2cba3599188 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:56.009051 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 178/351] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:56.010775 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d53540-0644-4ccb-aea3-4ed570853257 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:56.011071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d53540-0644-4ccb-aea3-4ed570853257 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:56.011338 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d53540-0644-4ccb-aea3-4ed570853257 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:56.015229 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 179/352] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:56.016397 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646cbd4-415a-4bf9-860f-d02e18f9df27 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:56.016684 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646cbd4-415a-4bf9-860f-d02e18f9df27 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:56.017504 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b646cbd4-415a-4bf9-860f-d02e18f9df27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:56.037557 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b646cbd4-415a-4bf9-860f-d02e18f9df27 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:56.038761 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b646cbd4-415a-4bf9-860f-d02e18f9df27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:56.110453 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922eb53-8cf3-4abd-8168-5d48ddfa5846 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:56.139494 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 180/353] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:56.140561 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646cbd4-415a-4bf9-860f-d02e18f9df27 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:56.153708 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163589e2-abdb-4413-90f9-57eb890aafbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:56.154447 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163589e2-abdb-4413-90f9-57eb890aafbb None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:56.154752 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163589e2-abdb-4413-90f9-57eb890aafbb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:56.155037 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163589e2-abdb-4413-90f9-57eb890aafbb None admin] PATH_INFO: `/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:56.156304 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163589e2-abdb-4413-90f9-57eb890aafbb None admin] RBAC: Authorizing `identity:get_project(project_id=79a2aa8010c54ad7ad7b27f08f4de74c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:56.157838 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163589e2-abdb-4413-90f9-57eb890aafbb None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:56.159736 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 174/354] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:55 2022] GET /identity/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:56.161020 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc172bb3-6f76-4210-875e-b76409b5288f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:56.161307 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc172bb3-6f76-4210-875e-b76409b5288f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:56.161713 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc172bb3-6f76-4210-875e-b76409b5288f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:56.182480 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-cc172bb3-6f76-4210-875e-b76409b5288f None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:56.183694 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cc172bb3-6f76-4210-875e-b76409b5288f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:56.233256 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:56.249500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b646cbd4-415a-4bf9-860f-d02e18f9df27 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:56.267988 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 175/355] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:56.269573 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80229be6-e1c3-43da-8a68-162865580bbd None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:56.269857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80229be6-e1c3-43da-8a68-162865580bbd None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:56.270129 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80229be6-e1c3-43da-8a68-162865580bbd None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:56.275337 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 176/356] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:56.276233 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80229be6-e1c3-43da-8a68-162865580bbd None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:56.278448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:56.289352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a29e6d-f3f6-4ff0-927e-d8fa350a4c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:56.290190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a29e6d-f3f6-4ff0-927e-d8fa350a4c99 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:56.297194 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a29e6d-f3f6-4ff0-927e-d8fa350a4c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:56.297194 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a29e6d-f3f6-4ff0-927e-d8fa350a4c99 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:56.297194 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a29e6d-f3f6-4ff0-927e-d8fa350a4c99 None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:56.297194 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a29e6d-f3f6-4ff0-927e-d8fa350a4c99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:56.297194 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a29e6d-f3f6-4ff0-927e-d8fa350a4c99 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:56.303792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 181/357] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Sep 2 09:13:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:56.305377 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b7702-981e-4529-87df-93e07f92a227 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:56.305810 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b7702-981e-4529-87df-93e07f92a227 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:56.306343 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7b7702-981e-4529-87df-93e07f92a227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:56.326957 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-dd7b7702-981e-4529-87df-93e07f92a227 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:56.328429 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-dd7b7702-981e-4529-87df-93e07f92a227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:56.394048 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80229be6-e1c3-43da-8a68-162865580bbd None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:56.400917 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 182/358] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:56.401986 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b7702-981e-4529-87df-93e07f92a227 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:56.455759 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32895a78-ed52-4380-a3d8-2710edc523e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:56.456831 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32895a78-ed52-4380-a3d8-2710edc523e6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:56.457295 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32895a78-ed52-4380-a3d8-2710edc523e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:56.457910 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32895a78-ed52-4380-a3d8-2710edc523e6 None admin] PATH_INFO: `/v3/groups/8a53ec1153c1430ea8a4ac281c27da8b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:56.459328 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32895a78-ed52-4380-a3d8-2710edc523e6 None admin] RBAC: Authorizing `identity:get_group(group_id=8a53ec1153c1430ea8a4ac281c27da8b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:56.460743 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32895a78-ed52-4380-a3d8-2710edc523e6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:56.463389 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 177/359] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Sep 2 09:13:56 2022] GET /identity/v3/groups/8a53ec1153c1430ea8a4ac281c27da8b => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:56.468012 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32895a78-ed52-4380-a3d8-2710edc523e6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:56.512203 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7b7702-981e-4529-87df-93e07f92a227 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:56.529536 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:56.539143 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:56.549683 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28221e6f-dc15-4f2c-8650-e55baeec8912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:56.550669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28221e6f-dc15-4f2c-8650-e55baeec8912 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:56.551175 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28221e6f-dc15-4f2c-8650-e55baeec8912 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:56.551656 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28221e6f-dc15-4f2c-8650-e55baeec8912 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:56.622224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32895a78-ed52-4380-a3d8-2710edc523e6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:56.628131 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28221e6f-dc15-4f2c-8650-e55baeec8912 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:56.629983 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28221e6f-dc15-4f2c-8650-e55baeec8912 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:56.705402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb5cf9-1586-4ec2-a9a5-14e17722282b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:56.705402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb5cf9-1586-4ec2-a9a5-14e17722282b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:56.705698 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb5cf9-1586-4ec2-a9a5-14e17722282b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:56.705929 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fb5cf9-1586-4ec2-a9a5-14e17722282b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:56.706559 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb5cf9-1586-4ec2-a9a5-14e17722282b None admin] RBAC: Adding query filter params (group.id=8a53ec1153c1430ea8a4ac281c27da8b, role.id=f2fe9fb31024471e9e2bea2b12715de6, scope.project.id=79a2aa8010c54ad7ad7b27f08f4de74c) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:56.706895 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb5cf9-1586-4ec2-a9a5-14e17722282b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:56.708332 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fb5cf9-1586-4ec2-a9a5-14e17722282b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:56.721701 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 178/360] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Sep 2 09:13:56 2022] GET /identity/v3/role_assignments?group.id=8a53ec1153c1430ea8a4ac281c27da8b&scope.project.id=79a2aa8010c54ad7ad7b27f08f4de74c&role.id=f2fe9fb31024471e9e2bea2b12715de6 => generated 260 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:13:56.722550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb5cf9-1586-4ec2-a9a5-14e17722282b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:56.740990 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 183/361] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:13:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:13:56.742203 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28221e6f-dc15-4f2c-8650-e55baeec8912 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:56.779257 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:56.864125 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28221e6f-dc15-4f2c-8650-e55baeec8912 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:56.916729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f68e12-4c23-4de1-b12c-8c1a894291e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:56.917370 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f68e12-4c23-4de1-b12c-8c1a894291e5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:56.917642 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f68e12-4c23-4de1-b12c-8c1a894291e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:56.917907 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f68e12-4c23-4de1-b12c-8c1a894291e5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:56.928867 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f68e12-4c23-4de1-b12c-8c1a894291e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:56.933789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-32f68e12-4c23-4de1-b12c-8c1a894291e5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 09:13:56.933789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:56.933789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:56.933789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:56.933789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:56.933789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:56.933789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:56.933789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:56.933789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:56.933789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:56.933789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:56.933789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:56.933789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:56.933789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:56.933789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:56.933789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:56.934685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:56.934685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:56.934685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:56.934685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:56.934685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:56.934685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:56.934685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:56.934685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:56.934685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:56.934685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:56.934685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:56.934685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:56.934685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:56.934685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:56.934685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:56.934685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:56.935837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:56.935837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:56.935837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:56.935837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:56.935837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:56.935837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:56.935837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:56.935837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:56.935837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:56.935837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:56.935837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 09:13:56.935837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:13:56.935837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 184/362] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Sep 2 09:13:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:56.935837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f68e12-4c23-4de1-b12c-8c1a894291e5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:56.967378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fb5cf9-1586-4ec2-a9a5-14e17722282b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:56.976306 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:57.014613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768dc9f-3bc8-4ba9-8e3e-716460d3c68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:57.015727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768dc9f-3bc8-4ba9-8e3e-716460d3c68d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:57.017119 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768dc9f-3bc8-4ba9-8e3e-716460d3c68d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:57.017119 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768dc9f-3bc8-4ba9-8e3e-716460d3c68d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:57.030422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768dc9f-3bc8-4ba9-8e3e-716460d3c68d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:57.037488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d768dc9f-3bc8-4ba9-8e3e-716460d3c68d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 09:13:57.037488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:57.037488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:57.037488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:57.037488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:57.037488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:57.037488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:57.037488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:57.037488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:57.037488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:57.037488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:57.037488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:57.037488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:57.037488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:57.037488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:57.037488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:57.038395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:57.038395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:57.038395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:57.038395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:57.038395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:57.038395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:57.038395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:57.038395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:57.038395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:57.038395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:57.038395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:57.038395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:57.038395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:57.038395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:57.038395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:57.038395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:57.039272 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:57.039272 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:57.039272 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:57.039272 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:57.039272 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:57.039272 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:57.039272 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:57.039272 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:57.039272 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:57.039272 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:57.039272 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 02 09:13:57.039272 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:13:57.039272 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 179/363] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:57.039935 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768dc9f-3bc8-4ba9-8e3e-716460d3c68d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:57.044300 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:57.066570 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f68e12-4c23-4de1-b12c-8c1a894291e5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:57.106893 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037d8099-1101-4dff-ae20-0d7f34c5e84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:57.107853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d8099-1101-4dff-ae20-0d7f34c5e84c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:57.108311 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d8099-1101-4dff-ae20-0d7f34c5e84c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:57.109171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d8099-1101-4dff-ae20-0d7f34c5e84c None admin] PATH_INFO: `/v3/projects/dac3a1a96e914a549133e73f03943618` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:57.110637 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037d8099-1101-4dff-ae20-0d7f34c5e84c None admin] RBAC: Authorizing `identity:get_project(project_id=dac3a1a96e914a549133e73f03943618)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:57.112512 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037d8099-1101-4dff-ae20-0d7f34c5e84c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:57.114698 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 185/364] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:56 2022] GET /identity/v3/projects/dac3a1a96e914a549133e73f03943618 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:57.121217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037d8099-1101-4dff-ae20-0d7f34c5e84c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:57.164417 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768dc9f-3bc8-4ba9-8e3e-716460d3c68d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:57.215625 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd66cf-0b8b-4786-beb6-33316fbf261c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:57.216301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd66cf-0b8b-4786-beb6-33316fbf261c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:57.216579 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd66cf-0b8b-4786-beb6-33316fbf261c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:57.216849 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd66cf-0b8b-4786-beb6-33316fbf261c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:57.232597 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecd66cf-0b8b-4786-beb6-33316fbf261c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:57.240916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-2ecd66cf-0b8b-4786-beb6-33316fbf261c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:13:57.240916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:57.240916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:57.240916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:57.240916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:57.240916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:57.240916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:57.240916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:57.240916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:57.240916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:57.240916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:57.240916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:57.240916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:57.240916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:57.240916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:57.240916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:57.241805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:57.241805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:57.241805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:57.241805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:57.241805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:57.241805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:57.241805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:57.241805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:57.241805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:57.241805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:57.241805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:57.241805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:57.241805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:57.241805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:57.241805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:57.241805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:57.242640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:57.242640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:57.242640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:57.242640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:57.242640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:57.242640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:57.242640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:57.242640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:57.242640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:57.242640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:57.242640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:13:57.242640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:13:57.242640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 180/365] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:13:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:57.243347 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd66cf-0b8b-4786-beb6-33316fbf261c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:57.275630 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:57.281769 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037d8099-1101-4dff-ae20-0d7f34c5e84c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:57.338727 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de28420-d549-418a-8ce1-60ba054e302b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:57.340131 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de28420-d549-418a-8ce1-60ba054e302b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:57.343388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de28420-d549-418a-8ce1-60ba054e302b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:57.343388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de28420-d549-418a-8ce1-60ba054e302b None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:57.344796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de28420-d549-418a-8ce1-60ba054e302b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:57.345192 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de28420-d549-418a-8ce1-60ba054e302b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:57.347240 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de28420-d549-418a-8ce1-60ba054e302b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:57.357358 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 186/366] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:13:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:57.358305 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de28420-d549-418a-8ce1-60ba054e302b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:57.369161 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:57.397405 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd66cf-0b8b-4786-beb6-33316fbf261c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:57.441863 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e92839-a697-42e9-bf04-fc3919d81bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:57.442737 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e92839-a697-42e9-bf04-fc3919d81bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:57.443186 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e92839-a697-42e9-bf04-fc3919d81bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:57.443617 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e92839-a697-42e9-bf04-fc3919d81bf7 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:57.444423 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e92839-a697-42e9-bf04-fc3919d81bf7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:57.444880 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e92839-a697-42e9-bf04-fc3919d81bf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:57.446934 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e92839-a697-42e9-bf04-fc3919d81bf7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:57.459041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 181/367] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Sep 2 09:13:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:57.460770 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e92839-a697-42e9-bf04-fc3919d81bf7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:57.485448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de28420-d549-418a-8ce1-60ba054e302b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:57.530407 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b090ff-f70e-4efb-8615-961032dd69f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:57.531313 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b090ff-f70e-4efb-8615-961032dd69f5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:57.532050 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b090ff-f70e-4efb-8615-961032dd69f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:57.532500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b090ff-f70e-4efb-8615-961032dd69f5 None admin] PATH_INFO: `/v3/groups/8a53ec1153c1430ea8a4ac281c27da8b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:57.534374 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b090ff-f70e-4efb-8615-961032dd69f5 None admin] RBAC: Authorizing `identity:get_group(group_id=8a53ec1153c1430ea8a4ac281c27da8b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:57.536813 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b090ff-f70e-4efb-8615-961032dd69f5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:57.539316 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 187/368] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Sep 2 09:13:57 2022] GET /identity/v3/groups/8a53ec1153c1430ea8a4ac281c27da8b => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:57.541020 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b090ff-f70e-4efb-8615-961032dd69f5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:57.574263 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:57.589918 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e92839-a697-42e9-bf04-fc3919d81bf7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:57.627029 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:57.628526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0209bcb-f241-4ee4-b776-d6c375e83838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:57.629391 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0209bcb-f241-4ee4-b776-d6c375e83838 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:57.629825 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0209bcb-f241-4ee4-b776-d6c375e83838 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:57.630335 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0209bcb-f241-4ee4-b776-d6c375e83838 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:57.631215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0209bcb-f241-4ee4-b776-d6c375e83838 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:57.631659 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0209bcb-f241-4ee4-b776-d6c375e83838 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:57.633165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0209bcb-f241-4ee4-b776-d6c375e83838 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:57.643509 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 182/369] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:57.644622 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0209bcb-f241-4ee4-b776-d6c375e83838 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:57.655680 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b090ff-f70e-4efb-8615-961032dd69f5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:57.691922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a8a62d-0beb-41c5-adda-33b249dae944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:57.693109 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8a62d-0beb-41c5-adda-33b249dae944 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:57.693487 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8a62d-0beb-41c5-adda-33b249dae944 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:57.693869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8a62d-0beb-41c5-adda-33b249dae944 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:57.755923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0209bcb-f241-4ee4-b776-d6c375e83838 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:57.768081 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a8a62d-0beb-41c5-adda-33b249dae944 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:57.769677 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a8a62d-0beb-41c5-adda-33b249dae944 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:57.796580 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd33149-1930-4f05-8d37-512a07550b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:57.797518 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd33149-1930-4f05-8d37-512a07550b69 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:57.797947 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd33149-1930-4f05-8d37-512a07550b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:57.798350 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd33149-1930-4f05-8d37-512a07550b69 None admin] PATH_INFO: `/v3/users/447c8e27c7f34d5493aad67ad5d60f11` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:57.799793 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd33149-1930-4f05-8d37-512a07550b69 None admin] RBAC: Authorizing `identity:get_user(user_id=447c8e27c7f34d5493aad67ad5d60f11)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:57.801412 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd33149-1930-4f05-8d37-512a07550b69 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:57.803271 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 183/370] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:13:57 2022] GET /identity/v3/users/447c8e27c7f34d5493aad67ad5d60f11 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:57.804174 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd33149-1930-4f05-8d37-512a07550b69 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:57.824753 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:57.856632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:57.889000 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 188/371] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:13:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:13:57.890230 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a8a62d-0beb-41c5-adda-33b249dae944 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:57.914542 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd33149-1930-4f05-8d37-512a07550b69 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:57.954383 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97279313-a2b4-4ba0-9833-58ad0e8a6b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:57.955083 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97279313-a2b4-4ba0-9833-58ad0e8a6b8d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:57.955367 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97279313-a2b4-4ba0-9833-58ad0e8a6b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:57.955693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97279313-a2b4-4ba0-9833-58ad0e8a6b8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:57.956428 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97279313-a2b4-4ba0-9833-58ad0e8a6b8d None admin] RBAC: Adding query filter params (group.id=8a53ec1153c1430ea8a4ac281c27da8b, role.id=efe6bdd5112340b48b262ceafbc19c04, scope.project.id=dac3a1a96e914a549133e73f03943618) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:57.956739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97279313-a2b4-4ba0-9833-58ad0e8a6b8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:57.958196 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97279313-a2b4-4ba0-9833-58ad0e8a6b8d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:57.964672 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 184/372] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Sep 2 09:13:57 2022] GET /identity/v3/role_assignments?group.id=8a53ec1153c1430ea8a4ac281c27da8b&scope.project.id=dac3a1a96e914a549133e73f03943618&role.id=efe6bdd5112340b48b262ceafbc19c04 => generated 260 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:57.965540 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97279313-a2b4-4ba0-9833-58ad0e8a6b8d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:58.047483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a8a62d-0beb-41c5-adda-33b249dae944 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:58.104419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497f8c47-2f2d-4c5c-9f27-20236ac3994a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:58.105057 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f8c47-2f2d-4c5c-9f27-20236ac3994a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:58.105384 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f8c47-2f2d-4c5c-9f27-20236ac3994a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:58.105661 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f8c47-2f2d-4c5c-9f27-20236ac3994a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:58.166472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:58.177770 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:58.191535 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97279313-a2b4-4ba0-9833-58ad0e8a6b8d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:58.211532 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497f8c47-2f2d-4c5c-9f27-20236ac3994a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:58.219056 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497f8c47-2f2d-4c5c-9f27-20236ac3994a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:58.248138 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2155112d-06ec-4aff-b767-a9dafe70b625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:58.248752 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155112d-06ec-4aff-b767-a9dafe70b625 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:58.249033 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155112d-06ec-4aff-b767-a9dafe70b625 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:58.249581 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155112d-06ec-4aff-b767-a9dafe70b625 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:58.250035 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2155112d-06ec-4aff-b767-a9dafe70b625 None admin] RBAC: Adding query filter params (role.id=b381d11646e54d1197676600202a6144, user.id=447c8e27c7f34d5493aad67ad5d60f11, scope.system=all) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:58.250346 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2155112d-06ec-4aff-b767-a9dafe70b625 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:58.251890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2155112d-06ec-4aff-b767-a9dafe70b625 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:58.284891 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 185/373] 10.222.0.57 () {56 vars in 1308 bytes} [Fri Sep 2 09:13:57 2022] GET /identity/v3/role_assignments?user.id=447c8e27c7f34d5493aad67ad5d60f11&scope.system=all&role.id=b381d11646e54d1197676600202a6144 => generated 510 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:58.285903 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2155112d-06ec-4aff-b767-a9dafe70b625 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:58.385483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 189/374] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:13:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:13:58.386634 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bdaa31-f29d-4d9c-a87c-e1423ee9568f None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:58.386949 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bdaa31-f29d-4d9c-a87c-e1423ee9568f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:58.387309 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bdaa31-f29d-4d9c-a87c-e1423ee9568f None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:58.388608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 190/375] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:58.392395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bdaa31-f29d-4d9c-a87c-e1423ee9568f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:58.414140 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2155112d-06ec-4aff-b767-a9dafe70b625 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:58.455758 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28951380-2f78-4c40-a5c1-2d3fb0f66414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:58.456419 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28951380-2f78-4c40-a5c1-2d3fb0f66414 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:58.456699 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28951380-2f78-4c40-a5c1-2d3fb0f66414 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:58.456970 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28951380-2f78-4c40-a5c1-2d3fb0f66414 None admin] PATH_INFO: `/v3/projects/dac3a1a96e914a549133e73f03943618` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:58.458312 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28951380-2f78-4c40-a5c1-2d3fb0f66414 None admin] RBAC: Authorizing `identity:get_project(project_id=dac3a1a96e914a549133e73f03943618)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:58.459827 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28951380-2f78-4c40-a5c1-2d3fb0f66414 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:58.461956 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 186/376] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:58 2022] GET /identity/v3/projects/dac3a1a96e914a549133e73f03943618 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:58.463725 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cb96a-5b1d-4cee-afdd-80cbdd2fc181 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:58.464007 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cb96a-5b1d-4cee-afdd-80cbdd2fc181 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:58.464276 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676cb96a-5b1d-4cee-afdd-80cbdd2fc181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:58.488042 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-676cb96a-5b1d-4cee-afdd-80cbdd2fc181 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:58.489276 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-676cb96a-5b1d-4cee-afdd-80cbdd2fc181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:58.493062 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:58.581476 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:58.590244 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 187/377] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:13:58.591151 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676cb96a-5b1d-4cee-afdd-80cbdd2fc181 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:58.662648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bdaa31-f29d-4d9c-a87c-e1423ee9568f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:58.769532 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2c91e-65bf-4c72-88a5-572ab0f72530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:58.770185 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2c91e-65bf-4c72-88a5-572ab0f72530 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:58.770467 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2c91e-65bf-4c72-88a5-572ab0f72530 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:58.770776 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2c91e-65bf-4c72-88a5-572ab0f72530 None admin] PATH_INFO: `/v3/projects/ae5820815add48cd913951058cc9251c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:58.775859 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d2c91e-65bf-4c72-88a5-572ab0f72530 None admin] RBAC: Authorizing `identity:get_project(project_id=ae5820815add48cd913951058cc9251c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:58.777319 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d2c91e-65bf-4c72-88a5-572ab0f72530 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:58.779426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 191/378] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:58 2022] GET /identity/v3/projects/ae5820815add48cd913951058cc9251c => generated 344 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:58.787746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2c91e-65bf-4c72-88a5-572ab0f72530 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:58.816011 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676cb96a-5b1d-4cee-afdd-80cbdd2fc181 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:58.870184 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533fd6b-9a99-4e9c-b762-95e272f3d811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:58.870861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533fd6b-9a99-4e9c-b762-95e272f3d811 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:58.873775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533fd6b-9a99-4e9c-b762-95e272f3d811 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:58.873775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533fd6b-9a99-4e9c-b762-95e272f3d811 None admin] PATH_INFO: `/v3/groups/8a53ec1153c1430ea8a4ac281c27da8b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:58.875748 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533fd6b-9a99-4e9c-b762-95e272f3d811 None admin] RBAC: Authorizing `identity:get_group(group_id=8a53ec1153c1430ea8a4ac281c27da8b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:58.882734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533fd6b-9a99-4e9c-b762-95e272f3d811 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:58.882734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 188/379] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Sep 2 09:13:58 2022] GET /identity/v3/groups/8a53ec1153c1430ea8a4ac281c27da8b => generated 226 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:58.888588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533fd6b-9a99-4e9c-b762-95e272f3d811 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:58.901999 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:59.006732 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2c91e-65bf-4c72-88a5-572ab0f72530 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:59.024724 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533fd6b-9a99-4e9c-b762-95e272f3d811 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:59.044131 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c26e65-0734-44a8-b44d-42316ab7e565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:59.044769 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c26e65-0734-44a8-b44d-42316ab7e565 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:59.045317 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c26e65-0734-44a8-b44d-42316ab7e565 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:59.045591 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c26e65-0734-44a8-b44d-42316ab7e565 None admin] PATH_INFO: `/v3/groups/b14338530049456aa7d40c06b40af01d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:59.053395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c26e65-0734-44a8-b44d-42316ab7e565 None admin] RBAC: Authorizing `identity:get_group(group_id=b14338530049456aa7d40c06b40af01d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:59.054879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c26e65-0734-44a8-b44d-42316ab7e565 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:59.058216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 192/380] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Sep 2 09:13:58 2022] GET /identity/v3/groups/b14338530049456aa7d40c06b40af01d => generated 229 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:59.059371 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c26e65-0734-44a8-b44d-42316ab7e565 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:59.072057 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:59.075466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eeee6e-f76f-4976-b733-69ac96d388fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:59.076447 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eeee6e-f76f-4976-b733-69ac96d388fb None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:59.076775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eeee6e-f76f-4976-b733-69ac96d388fb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:59.084738 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eeee6e-f76f-4976-b733-69ac96d388fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:59.084738 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eeee6e-f76f-4976-b733-69ac96d388fb None admin] RBAC: Adding query filter params (group.id=8a53ec1153c1430ea8a4ac281c27da8b, role.id=f2fe9fb31024471e9e2bea2b12715de6, scope.project.id=dac3a1a96e914a549133e73f03943618) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:59.084738 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eeee6e-f76f-4976-b733-69ac96d388fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:59.084738 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eeee6e-f76f-4976-b733-69ac96d388fb None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:59.088696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 189/381] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Sep 2 09:13:58 2022] GET /identity/v3/role_assignments?group.id=8a53ec1153c1430ea8a4ac281c27da8b&scope.project.id=dac3a1a96e914a549133e73f03943618&role.id=f2fe9fb31024471e9e2bea2b12715de6 => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:59.090103 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eeee6e-f76f-4976-b733-69ac96d388fb None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:59.197110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:59.233691 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eeee6e-f76f-4976-b733-69ac96d388fb None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:59.274198 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b560227e-d233-4a37-9c8d-753bfce4f023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:59.275283 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560227e-d233-4a37-9c8d-753bfce4f023 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:59.275563 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560227e-d233-4a37-9c8d-753bfce4f023 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:59.275835 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b560227e-d233-4a37-9c8d-753bfce4f023 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:59.276670 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b560227e-d233-4a37-9c8d-753bfce4f023 None admin] RBAC: Adding query filter params (group.id=b14338530049456aa7d40c06b40af01d, role.id=f886b8e8889549e896a8fb3426cce9e1, scope.project.id=ae5820815add48cd913951058cc9251c) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:59.277327 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b560227e-d233-4a37-9c8d-753bfce4f023 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:59.278887 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b560227e-d233-4a37-9c8d-753bfce4f023 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:59.285309 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 190/382] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Sep 2 09:13:59 2022] GET /identity/v3/role_assignments?group.id=b14338530049456aa7d40c06b40af01d&scope.project.id=ae5820815add48cd913951058cc9251c&role.id=f886b8e8889549e896a8fb3426cce9e1 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:59.286612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f71e5-bfea-41c5-9f91-ac3a7fd742ed None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:59.286978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f71e5-bfea-41c5-9f91-ac3a7fd742ed None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:59.287260 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f71e5-bfea-41c5-9f91-ac3a7fd742ed None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:59.288087 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c26e65-0734-44a8-b44d-42316ab7e565 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:59.288718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 191/383] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:13:59.295559 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd99ab-5e59-46f2-8b44-938653bbc58b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:59.295914 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd99ab-5e59-46f2-8b44-938653bbc58b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:59.296120 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd99ab-5e59-46f2-8b44-938653bbc58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:59.316521 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-cdfd99ab-5e59-46f2-8b44-938653bbc58b None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:13:59.318518 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cdfd99ab-5e59-46f2-8b44-938653bbc58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:13:59.341936 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a780478-9554-4a28-a95a-28e9dfba724e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:59.342641 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a780478-9554-4a28-a95a-28e9dfba724e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:59.343035 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a780478-9554-4a28-a95a-28e9dfba724e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:59.343303 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a780478-9554-4a28-a95a-28e9dfba724e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:59.363290 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a780478-9554-4a28-a95a-28e9dfba724e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:13:59.369899 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2a780478-9554-4a28-a95a-28e9dfba724e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:59.369899 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:13:59.369899 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:13:59.369899 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:13:59.369899 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:13:59.369899 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:13:59.369899 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:13:59.369899 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:13:59.369899 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:13:59.369899 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:13:59.369899 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:13:59.369899 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:13:59.369899 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:13:59.369899 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:13:59.369899 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:13:59.369899 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:13:59.370848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:13:59.370848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:13:59.370848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:13:59.370848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:13:59.370848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:13:59.370848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:13:59.370848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:13:59.370848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:13:59.370848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:13:59.370848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:13:59.370848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:13:59.370848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:13:59.370848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:13:59.370848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:13:59.370848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:13:59.370848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:13:59.371669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:13:59.371669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:13:59.371669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:13:59.371669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:13:59.371669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:13:59.371669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:13:59.371669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:13:59.371669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:13:59.371669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:13:59.371669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:13:59.371669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:13:59.371669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:13:59.377272 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 193/384] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:13:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:13:59.379015 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a780478-9554-4a28-a95a-28e9dfba724e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:59.468790 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:59.470383 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 192/385] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 09:13:59.515809 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a780478-9554-4a28-a95a-28e9dfba724e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:59.558622 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2179f78-1eab-4626-937d-01dbad2eadf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:59.559223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2179f78-1eab-4626-937d-01dbad2eadf9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:59.559513 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2179f78-1eab-4626-937d-01dbad2eadf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:59.559790 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2179f78-1eab-4626-937d-01dbad2eadf9 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:59.560472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2179f78-1eab-4626-937d-01dbad2eadf9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:13:59.560779 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2179f78-1eab-4626-937d-01dbad2eadf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:59.562166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2179f78-1eab-4626-937d-01dbad2eadf9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:59.571212 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 194/386] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Sep 2 09:13:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:59.577479 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd99ab-5e59-46f2-8b44-938653bbc58b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:59.672922 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:59.718741 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd99ab-5e59-46f2-8b44-938653bbc58b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:59.754863 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576a382-f130-49c7-abd3-1281b3590d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:59.755503 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576a382-f130-49c7-abd3-1281b3590d29 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:59.755782 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576a382-f130-49c7-abd3-1281b3590d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:59.756055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576a382-f130-49c7-abd3-1281b3590d29 None admin] PATH_INFO: `/v3/groups/8a53ec1153c1430ea8a4ac281c27da8b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:59.757169 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5576a382-f130-49c7-abd3-1281b3590d29 None admin] RBAC: Authorizing `identity:get_group(group_id=8a53ec1153c1430ea8a4ac281c27da8b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:59.758592 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5576a382-f130-49c7-abd3-1281b3590d29 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:59.761057 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 193/387] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Sep 2 09:13:59 2022] GET /identity/v3/groups/8a53ec1153c1430ea8a4ac281c27da8b => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:59.766471 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2179f78-1eab-4626-937d-01dbad2eadf9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:59.903795 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576a382-f130-49c7-abd3-1281b3590d29 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:13:59.906213 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2179f78-1eab-4626-937d-01dbad2eadf9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:13:59.939589 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:13:59.952237 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6ba49-d5a4-4fc3-8ce7-f72e6975df4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:13:59.956409 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6ba49-d5a4-4fc3-8ce7-f72e6975df4a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:13:59.956409 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6ba49-d5a4-4fc3-8ce7-f72e6975df4a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:13:59.956409 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6ba49-d5a4-4fc3-8ce7-f72e6975df4a None admin] PATH_INFO: `/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:13:59.956409 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6ba49-d5a4-4fc3-8ce7-f72e6975df4a None admin] RBAC: Authorizing `identity:get_project(project_id=79a2aa8010c54ad7ad7b27f08f4de74c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:13:59.956684 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6ba49-d5a4-4fc3-8ce7-f72e6975df4a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:13:59.958740 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 195/388] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:13:59 2022] GET /identity/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:13:59.965329 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6ba49-d5a4-4fc3-8ce7-f72e6975df4a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:00.116185 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6ba49-d5a4-4fc3-8ce7-f72e6975df4a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:00.156914 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0a9ab-8505-4a7e-a459-09bc94f1e92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:00.157593 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0a9ab-8505-4a7e-a459-09bc94f1e92c None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:00.157862 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0a9ab-8505-4a7e-a459-09bc94f1e92c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:00.158131 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0a9ab-8505-4a7e-a459-09bc94f1e92c None admin] PATH_INFO: `/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c/groups/8a53ec1153c1430ea8a4ac281c27da8b/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:00.160210 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0a9ab-8505-4a7e-a459-09bc94f1e92c None admin] RBAC: Authorizing `identity:create_grant(project_id=79a2aa8010c54ad7ad7b27f08f4de74c, group_id=8a53ec1153c1430ea8a4ac281c27da8b, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:00.160796 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576a382-f130-49c7-abd3-1281b3590d29 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:00.161628 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0a9ab-8505-4a7e-a459-09bc94f1e92c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:00.197418 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:00.203528 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d68ed-7090-4d14-9f98-1feda9c552e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:00.204167 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d68ed-7090-4d14-9f98-1feda9c552e4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:00.204535 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d68ed-7090-4d14-9f98-1feda9c552e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:00.204808 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d68ed-7090-4d14-9f98-1feda9c552e4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:00.216787 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2d68ed-7090-4d14-9f98-1feda9c552e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:00.221941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-df2d68ed-7090-4d14-9f98-1feda9c552e4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 09:14:00.221941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:00.221941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:00.221941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:00.221941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:00.221941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:00.221941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:00.221941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:00.221941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:00.221941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:00.221941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:00.221941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:00.221941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:00.221941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:00.221941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:00.221941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:00.222856 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:00.222856 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:00.222856 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:00.222856 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:00.222856 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:00.222856 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:00.222856 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:00.222856 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:00.222856 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:00.222856 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:00.222856 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:00.222856 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:00.222856 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:00.222856 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:00.222856 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:00.222856 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:00.223647 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:00.223647 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:00.223647 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:00.223647 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:00.223647 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:00.223647 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:00.223647 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:00.223647 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:00.223647 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:00.223647 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:00.223647 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 09:14:00.223647 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:00.223647 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 194/389] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Sep 2 09:13:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:00.234550 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 196/390] 10.222.0.57 () {58 vars in 1442 bytes} [Fri Sep 2 09:13:59 2022] PUT /identity/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c/groups/8a53ec1153c1430ea8a4ac281c27da8b/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:00.237505 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d68ed-7090-4d14-9f98-1feda9c552e4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:00.351858 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d68ed-7090-4d14-9f98-1feda9c552e4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:00.393865 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6558a49b-d327-4a9b-b372-cd2be6264b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:00.394466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558a49b-d327-4a9b-b372-cd2be6264b49 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:00.394766 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558a49b-d327-4a9b-b372-cd2be6264b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:00.395043 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558a49b-d327-4a9b-b372-cd2be6264b49 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:00.395715 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6558a49b-d327-4a9b-b372-cd2be6264b49 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:00.396015 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6558a49b-d327-4a9b-b372-cd2be6264b49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:00.397379 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6558a49b-d327-4a9b-b372-cd2be6264b49 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:00.407551 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 195/391] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:00.413314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0a9ab-8505-4a7e-a459-09bc94f1e92c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:00.499288 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:00.550685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0a9ab-8505-4a7e-a459-09bc94f1e92c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:00.561746 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b024a1-73f1-4315-93c8-b53aa536cbf4 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:00.562340 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b024a1-73f1-4315-93c8-b53aa536cbf4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:00.562881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b024a1-73f1-4315-93c8-b53aa536cbf4 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:00.564746 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 196/392] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:00.576642 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8510f-dd58-4ad1-8560-a6ba911e1979 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:00.576922 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8510f-dd58-4ad1-8560-a6ba911e1979 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:00.588156 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8510f-dd58-4ad1-8560-a6ba911e1979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:00.591138 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c827d922-02f1-40db-a68a-09a09975fc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:00.592046 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827d922-02f1-40db-a68a-09a09975fc97 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:00.592522 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827d922-02f1-40db-a68a-09a09975fc97 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:00.592984 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827d922-02f1-40db-a68a-09a09975fc97 None admin] PATH_INFO: `/v3/groups/8a53ec1153c1430ea8a4ac281c27da8b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:00.594282 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c827d922-02f1-40db-a68a-09a09975fc97 None admin] RBAC: Authorizing `identity:get_group(group_id=8a53ec1153c1430ea8a4ac281c27da8b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:00.595997 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c827d922-02f1-40db-a68a-09a09975fc97 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:00.598123 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 197/393] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Sep 2 09:14:00 2022] GET /identity/v3/groups/8a53ec1153c1430ea8a4ac281c27da8b => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:00.608830 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c827d922-02f1-40db-a68a-09a09975fc97 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:00.615968 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2bd8510f-dd58-4ad1-8560-a6ba911e1979 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:00.617158 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2bd8510f-dd58-4ad1-8560-a6ba911e1979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:00.676228 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:00.700950 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 197/394] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 02 09:14:00.766424 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c827d922-02f1-40db-a68a-09a09975fc97 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:00.802852 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:00.811054 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194eb99-5a0e-491f-8921-40c2018c027d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:00.811693 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194eb99-5a0e-491f-8921-40c2018c027d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:00.811971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194eb99-5a0e-491f-8921-40c2018c027d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:00.812243 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194eb99-5a0e-491f-8921-40c2018c027d None admin] PATH_INFO: `/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:00.813359 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a194eb99-5a0e-491f-8921-40c2018c027d None admin] RBAC: Authorizing `identity:get_project(project_id=79a2aa8010c54ad7ad7b27f08f4de74c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:00.814832 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a194eb99-5a0e-491f-8921-40c2018c027d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:00.816499 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 198/395] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:14:00 2022] GET /identity/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:14:00.822323 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd8510f-dd58-4ad1-8560-a6ba911e1979 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:00.951452 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd8510f-dd58-4ad1-8560-a6ba911e1979 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:00.997846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801dacf8-19d5-4cfb-b15f-692d241adebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:00.998515 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801dacf8-19d5-4cfb-b15f-692d241adebe None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:00.998814 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801dacf8-19d5-4cfb-b15f-692d241adebe None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:00.999092 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801dacf8-19d5-4cfb-b15f-692d241adebe None admin] PATH_INFO: `/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c/groups/8a53ec1153c1430ea8a4ac281c27da8b/roles/f2fe9fb31024471e9e2bea2b12715de6` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:01.001273 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801dacf8-19d5-4cfb-b15f-692d241adebe None admin] RBAC: Authorizing `identity:create_grant(project_id=79a2aa8010c54ad7ad7b27f08f4de74c, group_id=8a53ec1153c1430ea8a4ac281c27da8b, role_id=f2fe9fb31024471e9e2bea2b12715de6)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:01.002696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801dacf8-19d5-4cfb-b15f-692d241adebe None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:01.072441 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 198/396] 10.222.0.57 () {58 vars in 1442 bytes} [Fri Sep 2 09:14:00 2022] PUT /identity/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c/groups/8a53ec1153c1430ea8a4ac281c27da8b/roles/f2fe9fb31024471e9e2bea2b12715de6 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:01.147093 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194eb99-5a0e-491f-8921-40c2018c027d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:01.266965 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194eb99-5a0e-491f-8921-40c2018c027d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:01.312069 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e38c4-7a1e-430b-be8b-3e3d757c4c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:01.312702 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e38c4-7a1e-430b-be8b-3e3d757c4c5d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:01.312978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e38c4-7a1e-430b-be8b-3e3d757c4c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:01.313245 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e38c4-7a1e-430b-be8b-3e3d757c4c5d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:01.325435 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e38c4-7a1e-430b-be8b-3e3d757c4c5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:01.330697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-5e9e38c4-7a1e-430b-be8b-3e3d757c4c5d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:14:01.330697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:01.330697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:01.330697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:01.330697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:01.330697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:01.330697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:01.330697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:01.330697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:01.330697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:01.330697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:01.330697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:01.330697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:01.330697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:01.330697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:01.330697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:01.331601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:01.331601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:01.331601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:01.331601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:01.331601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:01.331601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:01.331601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:01.331601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:01.331601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:01.331601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:01.331601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:01.331601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:01.331601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:01.331601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:01.331601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:01.331601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:01.332400 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:01.332400 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:01.332400 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:01.332400 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:01.332400 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:01.332400 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:01.332400 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:01.332400 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:01.332400 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:01.332400 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:01.332400 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:14:01.332400 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:01.332400 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 199/397] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:14:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:01.338254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801dacf8-19d5-4cfb-b15f-692d241adebe None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:01.390557 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:01.455153 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801dacf8-19d5-4cfb-b15f-692d241adebe None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:01.505953 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee861c84-707d-4bc5-8615-f27d7019516f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:01.506571 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee861c84-707d-4bc5-8615-f27d7019516f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:01.506881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee861c84-707d-4bc5-8615-f27d7019516f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:01.507151 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee861c84-707d-4bc5-8615-f27d7019516f None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:01.507823 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee861c84-707d-4bc5-8615-f27d7019516f None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:01.508123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee861c84-707d-4bc5-8615-f27d7019516f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:01.509510 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee861c84-707d-4bc5-8615-f27d7019516f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:01.519116 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 199/398] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Sep 2 09:14:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:01.524872 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e38c4-7a1e-430b-be8b-3e3d757c4c5d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:01.584487 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:01.615776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a0fb5-b238-4f2d-9cd5-e2b37433df7a None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:01.616105 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a0fb5-b238-4f2d-9cd5-e2b37433df7a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:01.616386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a0fb5-b238-4f2d-9cd5-e2b37433df7a None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:01.617746 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 200/399] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:01.623175 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e18cc-793e-40c4-80fa-73bc10ac0fd8 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:01.623457 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e18cc-793e-40c4-80fa-73bc10ac0fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:01.626826 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e18cc-793e-40c4-80fa-73bc10ac0fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:01.640302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e38c4-7a1e-430b-be8b-3e3d757c4c5d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:01.659834 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f30e18cc-793e-40c4-80fa-73bc10ac0fd8 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:01.661030 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f30e18cc-793e-40c4-80fa-73bc10ac0fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:01.683919 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab3e64-19eb-4eaa-abb8-9d70b6d91252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:01.684555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab3e64-19eb-4eaa-abb8-9d70b6d91252 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:01.684832 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab3e64-19eb-4eaa-abb8-9d70b6d91252 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:01.685105 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acab3e64-19eb-4eaa-abb8-9d70b6d91252 None admin] PATH_INFO: `/v3/groups/8a53ec1153c1430ea8a4ac281c27da8b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:01.686223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab3e64-19eb-4eaa-abb8-9d70b6d91252 None admin] RBAC: Authorizing `identity:get_group(group_id=8a53ec1153c1430ea8a4ac281c27da8b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:01.687809 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acab3e64-19eb-4eaa-abb8-9d70b6d91252 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:01.689878 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 200/400] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Sep 2 09:14:01 2022] GET /identity/v3/groups/8a53ec1153c1430ea8a4ac281c27da8b => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:01.697176 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab3e64-19eb-4eaa-abb8-9d70b6d91252 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:01.795659 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 201/401] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 09:14:01.821791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acab3e64-19eb-4eaa-abb8-9d70b6d91252 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:01.848008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:01.867611 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dcc797-4e85-4c18-8f6e-154e63a4d9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:01.868852 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dcc797-4e85-4c18-8f6e-154e63a4d9f3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:01.869123 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dcc797-4e85-4c18-8f6e-154e63a4d9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:01.869395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dcc797-4e85-4c18-8f6e-154e63a4d9f3 None admin] PATH_INFO: `/v3/projects/dac3a1a96e914a549133e73f03943618` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:01.874738 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dcc797-4e85-4c18-8f6e-154e63a4d9f3 None admin] RBAC: Authorizing `identity:get_project(project_id=dac3a1a96e914a549133e73f03943618)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:01.875025 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dcc797-4e85-4c18-8f6e-154e63a4d9f3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:01.876927 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 201/402] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:14:01 2022] GET /identity/v3/projects/dac3a1a96e914a549133e73f03943618 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:01.884450 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e18cc-793e-40c4-80fa-73bc10ac0fd8 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:01.926756 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:02.003176 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e18cc-793e-40c4-80fa-73bc10ac0fd8 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:02.046327 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a32c4-2d03-4f44-b802-fcb8121dc29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:02.046940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a32c4-2d03-4f44-b802-fcb8121dc29c None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:02.047220 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a32c4-2d03-4f44-b802-fcb8121dc29c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:02.047492 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9a32c4-2d03-4f44-b802-fcb8121dc29c None admin] PATH_INFO: `/v3/projects/dac3a1a96e914a549133e73f03943618/groups/8a53ec1153c1430ea8a4ac281c27da8b/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:02.049402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9a32c4-2d03-4f44-b802-fcb8121dc29c None admin] RBAC: Authorizing `identity:create_grant(project_id=dac3a1a96e914a549133e73f03943618, group_id=8a53ec1153c1430ea8a4ac281c27da8b, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:02.050863 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9a32c4-2d03-4f44-b802-fcb8121dc29c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:02.110753 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86420f-da67-4ba9-b6a5-77a672a845c6 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:02.111318 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86420f-da67-4ba9-b6a5-77a672a845c6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:02.111766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86420f-da67-4ba9-b6a5-77a672a845c6 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:02.113312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 202/403] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:14:02.118585 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb879b7-6683-4204-8eae-6f44423e916a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:02.119115 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb879b7-6683-4204-8eae-6f44423e916a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:02.119556 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb879b7-6683-4204-8eae-6f44423e916a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:02.140148 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ccb879b7-6683-4204-8eae-6f44423e916a None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:02.141547 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ccb879b7-6683-4204-8eae-6f44423e916a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:02.202769 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 202/404] 10.222.0.57 () {58 vars in 1442 bytes} [Fri Sep 2 09:14:01 2022] PUT /identity/v3/projects/dac3a1a96e914a549133e73f03943618/groups/8a53ec1153c1430ea8a4ac281c27da8b/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:02.202769 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a32c4-2d03-4f44-b802-fcb8121dc29c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:02.228834 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 203/405] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 09:14:02.346000 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:02.352205 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9a32c4-2d03-4f44-b802-fcb8121dc29c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:02.403781 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5069462-629e-487b-9fba-1a5ad3b15869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:02.404420 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5069462-629e-487b-9fba-1a5ad3b15869 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:02.404743 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5069462-629e-487b-9fba-1a5ad3b15869 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:02.405018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5069462-629e-487b-9fba-1a5ad3b15869 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:02.420788 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5069462-629e-487b-9fba-1a5ad3b15869 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:02.425868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-f5069462-629e-487b-9fba-1a5ad3b15869 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 09:14:02.425868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:02.425868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:02.425868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:02.425868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:02.425868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:02.425868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:02.425868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:02.425868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:02.425868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:02.425868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:02.425868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:02.425868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:02.425868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:02.425868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:02.425868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:02.426797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:02.426797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:02.426797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:02.426797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:02.426797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:02.426797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:02.426797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:02.426797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:02.426797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:02.426797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:02.426797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:02.426797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:02.426797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:02.426797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:02.426797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:02.426797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:02.427638 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:02.427638 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:02.427638 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:02.427638 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:02.427638 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:02.427638 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:02.427638 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:02.427638 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:02.427638 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:02.427638 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:02.427638 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 09:14:02.427638 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:02.428260 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 203/406] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Sep 2 09:14:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:02.433403 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb879b7-6683-4204-8eae-6f44423e916a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:02.575631 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb879b7-6683-4204-8eae-6f44423e916a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:02.604857 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:02.627825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e5367-452a-4168-9ee6-c81f09abf005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:02.628446 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e5367-452a-4168-9ee6-c81f09abf005 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:02.628723 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e5367-452a-4168-9ee6-c81f09abf005 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:02.628993 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e5367-452a-4168-9ee6-c81f09abf005 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:02.629665 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e5367-452a-4168-9ee6-c81f09abf005 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:02.629973 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e5367-452a-4168-9ee6-c81f09abf005 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:02.631693 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e5367-452a-4168-9ee6-c81f09abf005 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:02.643873 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 204/407] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:02.650030 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5069462-629e-487b-9fba-1a5ad3b15869 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:02.752943 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e5367-452a-4168-9ee6-c81f09abf005 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:02.803624 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5069462-629e-487b-9fba-1a5ad3b15869 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:02.866937 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6c880-c454-45f7-bbc1-e9612d961fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:02.867503 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6c880-c454-45f7-bbc1-e9612d961fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:02.867784 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6c880-c454-45f7-bbc1-e9612d961fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:02.868589 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6c880-c454-45f7-bbc1-e9612d961fe7 None admin] PATH_INFO: `/v3/groups/8a53ec1153c1430ea8a4ac281c27da8b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:02.870972 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c6c880-c454-45f7-bbc1-e9612d961fe7 None admin] RBAC: Authorizing `identity:get_group(group_id=8a53ec1153c1430ea8a4ac281c27da8b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:02.872416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c6c880-c454-45f7-bbc1-e9612d961fe7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:02.879622 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 204/408] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Sep 2 09:14:02 2022] GET /identity/v3/groups/8a53ec1153c1430ea8a4ac281c27da8b => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:02.880812 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6c880-c454-45f7-bbc1-e9612d961fe7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:02.932117 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:02.933811 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e5367-452a-4168-9ee6-c81f09abf005 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:02.981781 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8091f-6465-4407-b171-587112738c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:02.983138 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8091f-6465-4407-b171-587112738c6f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:02.983617 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8091f-6465-4407-b171-587112738c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:02.984353 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8091f-6465-4407-b171-587112738c6f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:03.002061 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a8091f-6465-4407-b171-587112738c6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:03.006304 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6c880-c454-45f7-bbc1-e9612d961fe7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:03.009678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-d7a8091f-6465-4407-b171-587112738c6f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:14:03.009678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:03.009678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:03.009678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:03.009678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:03.009678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:03.009678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:03.009678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:03.009678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:03.009678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:03.009678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:03.009678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:03.009678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:03.009678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:03.009678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:03.009678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:03.010584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:03.010584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:03.010584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:03.010584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:03.010584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:03.010584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:03.010584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:03.010584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:03.010584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:03.010584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:03.010584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:03.010584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:03.010584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:03.010584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:03.010584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:03.010584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:03.011397 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:03.011397 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:03.011397 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:03.011397 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:03.011397 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:03.011397 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:03.011397 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:03.011397 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:03.011397 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:03.011397 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:03.011397 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:14:03.011397 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:03.018560 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 205/409] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:14:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:14:03.018560 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb1620-b868-4ee2-a9c5-b20a658091f4 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:03.018560 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb1620-b868-4ee2-a9c5-b20a658091f4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:03.018560 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebb1620-b868-4ee2-a9c5-b20a658091f4 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:03.020564 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 206/410] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:03.021664 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb1620-b868-4ee2-a9c5-b20a658091f4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:03.048753 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201df6fd-79bb-46a6-b0af-29be43b4caec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:03.049387 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201df6fd-79bb-46a6-b0af-29be43b4caec None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:03.049665 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201df6fd-79bb-46a6-b0af-29be43b4caec None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:03.049938 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201df6fd-79bb-46a6-b0af-29be43b4caec None admin] PATH_INFO: `/v3/projects/dac3a1a96e914a549133e73f03943618` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:03.051085 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201df6fd-79bb-46a6-b0af-29be43b4caec None admin] RBAC: Authorizing `identity:get_project(project_id=dac3a1a96e914a549133e73f03943618)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:03.052685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201df6fd-79bb-46a6-b0af-29be43b4caec None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:03.059371 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 205/411] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:14:02 2022] GET /identity/v3/projects/dac3a1a96e914a549133e73f03943618 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:14:03.060613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e3bfa-5dba-46f6-9a77-b336b33fc589 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:03.060898 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e3bfa-5dba-46f6-9a77-b336b33fc589 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:03.061171 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e3bfa-5dba-46f6-9a77-b336b33fc589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:03.079194 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:03.081031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f70e3bfa-5dba-46f6-9a77-b336b33fc589 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:03.082417 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f70e3bfa-5dba-46f6-9a77-b336b33fc589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:03.151874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebb1620-b868-4ee2-a9c5-b20a658091f4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:03.158737 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 206/412] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:03.158737 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e3bfa-5dba-46f6-9a77-b336b33fc589 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:03.197343 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:03.220103 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe947fe-0bf6-4f69-931a-b54a3355f58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:03.220923 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe947fe-0bf6-4f69-931a-b54a3355f58b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:03.222251 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe947fe-0bf6-4f69-931a-b54a3355f58b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:03.222688 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe947fe-0bf6-4f69-931a-b54a3355f58b None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:03.224183 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe947fe-0bf6-4f69-931a-b54a3355f58b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:03.225862 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe947fe-0bf6-4f69-931a-b54a3355f58b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:03.227425 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe947fe-0bf6-4f69-931a-b54a3355f58b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:03.243764 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 207/413] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:14:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:03.248524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe947fe-0bf6-4f69-931a-b54a3355f58b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:03.343491 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70e3bfa-5dba-46f6-9a77-b336b33fc589 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:03.405490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c8a10-5e76-458f-9536-0da74f958f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:03.406081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c8a10-5e76-458f-9536-0da74f958f77 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:03.406351 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c8a10-5e76-458f-9536-0da74f958f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:03.406624 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c8a10-5e76-458f-9536-0da74f958f77 None admin] PATH_INFO: `/v3/projects/dac3a1a96e914a549133e73f03943618/groups/8a53ec1153c1430ea8a4ac281c27da8b/roles/f2fe9fb31024471e9e2bea2b12715de6` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:03.409203 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c8a10-5e76-458f-9536-0da74f958f77 None admin] RBAC: Authorizing `identity:create_grant(project_id=dac3a1a96e914a549133e73f03943618, group_id=8a53ec1153c1430ea8a4ac281c27da8b, role_id=f2fe9fb31024471e9e2bea2b12715de6)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:03.410652 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c8a10-5e76-458f-9536-0da74f958f77 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:03.415640 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe947fe-0bf6-4f69-931a-b54a3355f58b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:03.422746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:03.453055 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d752286-3a22-4b9a-8dc0-93349a666422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:03.454791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d752286-3a22-4b9a-8dc0-93349a666422 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:03.454791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d752286-3a22-4b9a-8dc0-93349a666422 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:03.454791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d752286-3a22-4b9a-8dc0-93349a666422 None admin] PATH_INFO: `/v3/groups/b14338530049456aa7d40c06b40af01d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:03.455729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d752286-3a22-4b9a-8dc0-93349a666422 None admin] RBAC: Authorizing `identity:get_group(group_id=b14338530049456aa7d40c06b40af01d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:03.457212 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d752286-3a22-4b9a-8dc0-93349a666422 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:03.459350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 208/414] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Sep 2 09:14:03 2022] GET /identity/v3/groups/b14338530049456aa7d40c06b40af01d => generated 229 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:03.462265 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37800825-4dec-48a0-985e-ae10c5029b13 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:03.462547 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37800825-4dec-48a0-985e-ae10c5029b13 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:03.462872 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37800825-4dec-48a0-985e-ae10c5029b13 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:03.464353 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 209/415] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:14:03.465049 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37800825-4dec-48a0-985e-ae10c5029b13 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:03.484687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 207/416] 10.222.0.57 () {58 vars in 1442 bytes} [Fri Sep 2 09:14:03 2022] PUT /identity/v3/projects/dac3a1a96e914a549133e73f03943618/groups/8a53ec1153c1430ea8a4ac281c27da8b/roles/f2fe9fb31024471e9e2bea2b12715de6 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:03.486318 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2b091-58b2-417c-8834-91ee572f2a40 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:03.486889 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2b091-58b2-417c-8834-91ee572f2a40 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:03.487389 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2b091-58b2-417c-8834-91ee572f2a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:03.512590 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-57f2b091-58b2-417c-8834-91ee572f2a40 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:03.514301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-57f2b091-58b2-417c-8834-91ee572f2a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:03.606753 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 208/417] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:03.610747 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2b091-58b2-417c-8834-91ee572f2a40 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:03.623614 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:03.645185 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37800825-4dec-48a0-985e-ae10c5029b13 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:03.697345 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5519be0-869b-44f5-afbd-b686999de816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:03.697893 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5519be0-869b-44f5-afbd-b686999de816 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:03.698167 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5519be0-869b-44f5-afbd-b686999de816 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:03.698491 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5519be0-869b-44f5-afbd-b686999de816 None admin] PATH_INFO: `/v3/projects/ae5820815add48cd913951058cc9251c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:03.699641 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5519be0-869b-44f5-afbd-b686999de816 None admin] RBAC: Authorizing `identity:get_project(project_id=ae5820815add48cd913951058cc9251c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:03.701524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5519be0-869b-44f5-afbd-b686999de816 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:03.706818 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 210/418] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:14:03 2022] GET /identity/v3/projects/ae5820815add48cd913951058cc9251c => generated 344 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:03.709968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5519be0-869b-44f5-afbd-b686999de816 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:03.720726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2b091-58b2-417c-8834-91ee572f2a40 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:03.756917 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:03.768494 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941ded6-c6d8-40d9-8fc5-4231eac810ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:03.769467 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941ded6-c6d8-40d9-8fc5-4231eac810ff None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:03.769879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941ded6-c6d8-40d9-8fc5-4231eac810ff None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:03.770280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941ded6-c6d8-40d9-8fc5-4231eac810ff None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:03.794637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941ded6-c6d8-40d9-8fc5-4231eac810ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:03.799588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-6941ded6-c6d8-40d9-8fc5-4231eac810ff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:14:03.799588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:03.799588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:03.799588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:03.799588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:03.799588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:03.799588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:03.799588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:03.799588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:03.799588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:03.799588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:03.799588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:03.799588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:03.799588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:03.799588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:03.799588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:03.800687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:03.800687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:03.800687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:03.800687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:03.800687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:03.800687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:03.800687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:03.800687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:03.800687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:03.800687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:03.800687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:03.800687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:03.800687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:03.800687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:03.800687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:03.800687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:03.801489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:03.801489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:03.801489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:03.801489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:03.801489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:03.801489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:03.801489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:03.801489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:03.801489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:03.801489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:03.801489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:14:03.801489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:03.801489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 209/419] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:14:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:03.816463 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941ded6-c6d8-40d9-8fc5-4231eac810ff None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:03.842408 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5519be0-869b-44f5-afbd-b686999de816 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:03.890485 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba01a2-1386-4b5b-a091-5f6f93bb62f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:03.891295 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba01a2-1386-4b5b-a091-5f6f93bb62f7 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:03.891697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba01a2-1386-4b5b-a091-5f6f93bb62f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:03.891981 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba01a2-1386-4b5b-a091-5f6f93bb62f7 None admin] PATH_INFO: `/v3/projects/ae5820815add48cd913951058cc9251c/groups/b14338530049456aa7d40c06b40af01d/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:03.893934 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba01a2-1386-4b5b-a091-5f6f93bb62f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae5820815add48cd913951058cc9251c, group_id=b14338530049456aa7d40c06b40af01d, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:03.895416 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba01a2-1386-4b5b-a091-5f6f93bb62f7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:03.927965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:03.959088 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941ded6-c6d8-40d9-8fc5-4231eac810ff None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:03.975262 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 211/420] 10.222.0.57 () {58 vars in 1442 bytes} [Fri Sep 2 09:14:03 2022] PUT /identity/v3/projects/ae5820815add48cd913951058cc9251c/groups/b14338530049456aa7d40c06b40af01d/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:04.006477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374f4a1-d07d-4d7d-8db2-b609b8ebddba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:04.007179 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374f4a1-d07d-4d7d-8db2-b609b8ebddba None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:04.007459 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374f4a1-d07d-4d7d-8db2-b609b8ebddba None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:04.007786 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d374f4a1-d07d-4d7d-8db2-b609b8ebddba None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:04.008459 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374f4a1-d07d-4d7d-8db2-b609b8ebddba None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:04.009386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374f4a1-d07d-4d7d-8db2-b609b8ebddba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:04.010864 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d374f4a1-d07d-4d7d-8db2-b609b8ebddba None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:04.023064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba01a2-1386-4b5b-a091-5f6f93bb62f7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:04.029080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 210/421] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Sep 2 09:14:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:04.040363 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374f4a1-d07d-4d7d-8db2-b609b8ebddba None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:04.194195 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:04.200136 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba01a2-1386-4b5b-a091-5f6f93bb62f7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:04.226998 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d374f4a1-d07d-4d7d-8db2-b609b8ebddba None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:04.254031 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982eb816-44fd-4990-87a3-2ec4707d0dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:04.255531 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982eb816-44fd-4990-87a3-2ec4707d0dbc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:04.255957 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982eb816-44fd-4990-87a3-2ec4707d0dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:04.256629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982eb816-44fd-4990-87a3-2ec4707d0dbc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:04.273382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982eb816-44fd-4990-87a3-2ec4707d0dbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:04.275779 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0b90d-d39e-41ba-be57-bfc76f1af71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:04.276508 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0b90d-d39e-41ba-be57-bfc76f1af71e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:04.276898 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0b90d-d39e-41ba-be57-bfc76f1af71e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:04.277301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0b90d-d39e-41ba-be57-bfc76f1af71e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:04.280512 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:04.283508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-982eb816-44fd-4990-87a3-2ec4707d0dbc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 09:14:04.283508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:04.283508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:04.283508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:04.283508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:04.283508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:04.283508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:04.283508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:04.283508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:04.283508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:04.283508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:04.283508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:04.283508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:04.283508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:04.283508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:04.283508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:04.284433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:04.284433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:04.284433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:04.284433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:04.284433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:04.284433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:04.284433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:04.284433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:04.284433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:04.284433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:04.284433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:04.284433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:04.284433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:04.284433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:04.284433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:04.284433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:04.285210 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:04.285210 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:04.285210 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:04.285210 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:04.285210 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:04.285210 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:04.285210 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:04.285210 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:04.285210 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:04.285210 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:04.285210 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 09:14:04.285210 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:04.288449 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 212/422] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Sep 2 09:14:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:04.298176 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982eb816-44fd-4990-87a3-2ec4707d0dbc None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:04.406340 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0b90d-d39e-41ba-be57-bfc76f1af71e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:04.408458 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0b90d-d39e-41ba-be57-bfc76f1af71e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:04.479117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982eb816-44fd-4990-87a3-2ec4707d0dbc None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:04.516769 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb666fc-5d05-49d2-8853-1cdcf6544958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:04.517710 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb666fc-5d05-49d2-8853-1cdcf6544958 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:04.518138 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb666fc-5d05-49d2-8853-1cdcf6544958 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:04.518553 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb666fc-5d05-49d2-8853-1cdcf6544958 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:04.519500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb666fc-5d05-49d2-8853-1cdcf6544958 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:04.520055 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb666fc-5d05-49d2-8853-1cdcf6544958 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:04.521621 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb666fc-5d05-49d2-8853-1cdcf6544958 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:04.536069 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 213/423] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:14:04.537703 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14968692-ad0d-41dc-9da2-4284e82dca0b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:04.538254 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14968692-ad0d-41dc-9da2-4284e82dca0b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:04.538868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14968692-ad0d-41dc-9da2-4284e82dca0b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:04.541155 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 214/424] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:04.542136 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14968692-ad0d-41dc-9da2-4284e82dca0b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:04.558795 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 211/425] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:14:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 02 09:14:04.560377 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2655e97-6263-42d0-911e-22ffe680384e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:04.560855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2655e97-6263-42d0-911e-22ffe680384e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:04.561326 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2655e97-6263-42d0-911e-22ffe680384e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:04.574111 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:04.585382 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d2655e97-6263-42d0-911e-22ffe680384e None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:04.587497 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d2655e97-6263-42d0-911e-22ffe680384e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:04.625546 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:04.671421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14968692-ad0d-41dc-9da2-4284e82dca0b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:04.687042 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 212/426] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:04.688370 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2655e97-6263-42d0-911e-22ffe680384e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:04.719123 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3d6b6-6f4e-4c9c-8ffe-7e73e9fb3eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:04.720575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3d6b6-6f4e-4c9c-8ffe-7e73e9fb3eee None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:04.720575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3d6b6-6f4e-4c9c-8ffe-7e73e9fb3eee None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:04.720575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d3d6b6-6f4e-4c9c-8ffe-7e73e9fb3eee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:04.846959 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3d6b6-6f4e-4c9c-8ffe-7e73e9fb3eee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:04.846959 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d3d6b6-6f4e-4c9c-8ffe-7e73e9fb3eee None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:04.863429 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:04.912608 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2655e97-6263-42d0-911e-22ffe680384e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:04.951721 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 215/427] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:14:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:14:04.961390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3d6b6-6f4e-4c9c-8ffe-7e73e9fb3eee None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:04.993827 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d192142-58e3-42bf-b8c9-757cb284bbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:04.994582 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d192142-58e3-42bf-b8c9-757cb284bbf3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:04.994884 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d192142-58e3-42bf-b8c9-757cb284bbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:04.995153 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d192142-58e3-42bf-b8c9-757cb284bbf3 None admin] PATH_INFO: `/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:04.996518 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d192142-58e3-42bf-b8c9-757cb284bbf3 None admin] RBAC: Authorizing `identity:get_project(project_id=79a2aa8010c54ad7ad7b27f08f4de74c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:04.999691 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d192142-58e3-42bf-b8c9-757cb284bbf3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:05.004469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 213/428] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:14:04 2022] GET /identity/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c => generated 298 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:05.011662 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d192142-58e3-42bf-b8c9-757cb284bbf3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:05.070171 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:05.089335 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d3d6b6-6f4e-4c9c-8ffe-7e73e9fb3eee None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:05.112286 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:05.131970 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7577d5e6-18f9-432a-ac48-dee6f1e4891c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:05.132876 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577d5e6-18f9-432a-ac48-dee6f1e4891c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:05.133350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577d5e6-18f9-432a-ac48-dee6f1e4891c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:05.134567 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7577d5e6-18f9-432a-ac48-dee6f1e4891c None admin] PATH_INFO: `/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:05.136478 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7577d5e6-18f9-432a-ac48-dee6f1e4891c None admin] RBAC: Authorizing `identity:get_project(project_id=79a2aa8010c54ad7ad7b27f08f4de74c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:05.138360 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7577d5e6-18f9-432a-ac48-dee6f1e4891c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:05.146382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 216/429] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:14:04 2022] GET /identity/v3/projects/79a2aa8010c54ad7ad7b27f08f4de74c => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:05.147567 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7577d5e6-18f9-432a-ac48-dee6f1e4891c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:05.181339 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d192142-58e3-42bf-b8c9-757cb284bbf3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:05.228922 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a243945-5e8d-483d-b0eb-30f8f79d35e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:05.230571 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a243945-5e8d-483d-b0eb-30f8f79d35e2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:05.231070 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a243945-5e8d-483d-b0eb-30f8f79d35e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:05.231528 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a243945-5e8d-483d-b0eb-30f8f79d35e2 None admin] PATH_INFO: `/v3/groups/8a53ec1153c1430ea8a4ac281c27da8b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:05.232873 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a243945-5e8d-483d-b0eb-30f8f79d35e2 None admin] RBAC: Authorizing `identity:get_group(group_id=8a53ec1153c1430ea8a4ac281c27da8b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:05.235308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a243945-5e8d-483d-b0eb-30f8f79d35e2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:05.238771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 214/430] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Sep 2 09:14:05 2022] GET /identity/v3/groups/8a53ec1153c1430ea8a4ac281c27da8b => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:05.239944 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a243945-5e8d-483d-b0eb-30f8f79d35e2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:05.281346 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7577d5e6-18f9-432a-ac48-dee6f1e4891c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:05.320305 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957ba46-1eee-460f-b220-5b6cff2b3f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:05.321236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957ba46-1eee-460f-b220-5b6cff2b3f44 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:05.321612 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957ba46-1eee-460f-b220-5b6cff2b3f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:05.322000 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957ba46-1eee-460f-b220-5b6cff2b3f44 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:05.335505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d957ba46-1eee-460f-b220-5b6cff2b3f44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:05.342544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-d957ba46-1eee-460f-b220-5b6cff2b3f44 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:14:05.342544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:05.342544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:05.342544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:05.342544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:05.342544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:05.342544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:05.342544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:05.342544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:05.342544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:05.342544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:05.342544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:05.342544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:05.342544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:05.342544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:05.342544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:05.343729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:05.343729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:05.343729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:05.343729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:05.343729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:05.343729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:05.343729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:05.343729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:05.343729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:05.343729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:05.343729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:05.343729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:05.343729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:05.343729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:05.343729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:05.343729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:05.344786 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:05.344786 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:05.344786 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:05.344786 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:05.344786 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:05.344786 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:05.344786 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:05.344786 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:05.344786 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:05.344786 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:05.344786 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 02 09:14:05.344786 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:05.351069 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 217/431] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:14:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:05.351069 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957ba46-1eee-460f-b220-5b6cff2b3f44 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:05.367705 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:05.371380 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a243945-5e8d-483d-b0eb-30f8f79d35e2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:05.407813 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:05.420820 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6026360c-66c5-44fb-b58b-cba885073a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:05.421441 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6026360c-66c5-44fb-b58b-cba885073a82 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:05.421721 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6026360c-66c5-44fb-b58b-cba885073a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:05.422174 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6026360c-66c5-44fb-b58b-cba885073a82 None admin] PATH_INFO: `/v3/groups/8a53ec1153c1430ea8a4ac281c27da8b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:05.423629 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6026360c-66c5-44fb-b58b-cba885073a82 None admin] RBAC: Authorizing `identity:get_group(group_id=8a53ec1153c1430ea8a4ac281c27da8b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:05.425352 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6026360c-66c5-44fb-b58b-cba885073a82 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:05.430743 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 215/432] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Sep 2 09:14:05 2022] GET /identity/v3/groups/8a53ec1153c1430ea8a4ac281c27da8b => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:05.430743 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6026360c-66c5-44fb-b58b-cba885073a82 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:05.471645 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957ba46-1eee-460f-b220-5b6cff2b3f44 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:05.509255 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61092ef-2dd3-482b-ac3e-f53a0c09ed0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:05.510317 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61092ef-2dd3-482b-ac3e-f53a0c09ed0e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:05.510829 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61092ef-2dd3-482b-ac3e-f53a0c09ed0e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:05.511320 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61092ef-2dd3-482b-ac3e-f53a0c09ed0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:05.512337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61092ef-2dd3-482b-ac3e-f53a0c09ed0e None admin] RBAC: Adding query filter params (group.id=8a53ec1153c1430ea8a4ac281c27da8b, role.id=efe6bdd5112340b48b262ceafbc19c04, scope.project.id=79a2aa8010c54ad7ad7b27f08f4de74c) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:05.512982 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61092ef-2dd3-482b-ac3e-f53a0c09ed0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:05.514614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61092ef-2dd3-482b-ac3e-f53a0c09ed0e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:05.521674 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 218/433] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Sep 2 09:14:05 2022] GET /identity/v3/role_assignments?group.id=8a53ec1153c1430ea8a4ac281c27da8b&scope.project.id=79a2aa8010c54ad7ad7b27f08f4de74c&role.id=efe6bdd5112340b48b262ceafbc19c04 => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:05.523021 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61092ef-2dd3-482b-ac3e-f53a0c09ed0e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:05.557477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6026360c-66c5-44fb-b58b-cba885073a82 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:05.603269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af0ac3-0838-4867-8f07-ea534d99751b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:05.603884 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af0ac3-0838-4867-8f07-ea534d99751b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:05.604164 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af0ac3-0838-4867-8f07-ea534d99751b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:05.604459 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af0ac3-0838-4867-8f07-ea534d99751b None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:05.605609 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af0ac3-0838-4867-8f07-ea534d99751b None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:05.605913 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af0ac3-0838-4867-8f07-ea534d99751b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:05.607474 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73af0ac3-0838-4867-8f07-ea534d99751b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:05.620304 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 216/434] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Sep 2 09:14:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:05.630274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af0ac3-0838-4867-8f07-ea534d99751b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:05.677398 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:05.737013 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61092ef-2dd3-482b-ac3e-f53a0c09ed0e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:05.760793 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:05.794902 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af0ac3-0838-4867-8f07-ea534d99751b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:05.801489 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5657d3-b9a5-4929-ba51-bf245e2da60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:05.803088 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5657d3-b9a5-4929-ba51-bf245e2da60e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:05.803898 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5657d3-b9a5-4929-ba51-bf245e2da60e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:05.806178 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5657d3-b9a5-4929-ba51-bf245e2da60e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:05.806178 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5657d3-b9a5-4929-ba51-bf245e2da60e None admin] RBAC: Adding query filter params (group.id=8a53ec1153c1430ea8a4ac281c27da8b, role.id=f2fe9fb31024471e9e2bea2b12715de6, scope.project.id=79a2aa8010c54ad7ad7b27f08f4de74c) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:05.806503 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5657d3-b9a5-4929-ba51-bf245e2da60e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:05.808686 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5657d3-b9a5-4929-ba51-bf245e2da60e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:05.821171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 219/435] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Sep 2 09:14:05 2022] GET /identity/v3/role_assignments?group.id=8a53ec1153c1430ea8a4ac281c27da8b&scope.project.id=79a2aa8010c54ad7ad7b27f08f4de74c&role.id=f2fe9fb31024471e9e2bea2b12715de6 => generated 611 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:05.826494 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1da97-99bf-4711-a4ed-58be99725917 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:05.826854 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1da97-99bf-4711-a4ed-58be99725917 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:05.827129 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e1da97-99bf-4711-a4ed-58be99725917 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:05.828464 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 220/436] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:05.836409 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8de234-8d40-48d3-b791-8b00811e80d4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:05.837219 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8de234-8d40-48d3-b791-8b00811e80d4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:05.837495 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8de234-8d40-48d3-b791-8b00811e80d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:05.855605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf017e6d-2eb3-4c81-a129-a289dd0785f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:05.856869 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf017e6d-2eb3-4c81-a129-a289dd0785f9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:05.857382 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf017e6d-2eb3-4c81-a129-a289dd0785f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:05.857839 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf017e6d-2eb3-4c81-a129-a289dd0785f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:05.887941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-dc8de234-8d40-48d3-b791-8b00811e80d4 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:05.889320 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-dc8de234-8d40-48d3-b791-8b00811e80d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:05.962039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf017e6d-2eb3-4c81-a129-a289dd0785f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:05.964528 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf017e6d-2eb3-4c81-a129-a289dd0785f9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:05.982744 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 221/437] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:06.009492 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:06.086140 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 217/438] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:14:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:14:06.092771 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8de234-8d40-48d3-b791-8b00811e80d4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:06.173302 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796f8db-32e9-4ee7-a8b8-55556fbcbcf6 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:06.173593 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796f8db-32e9-4ee7-a8b8-55556fbcbcf6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:06.173765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796f8db-32e9-4ee7-a8b8-55556fbcbcf6 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:06.175402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 218/439] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:14:06.184621 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff4196-5535-4d38-abce-72faf6043ab3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:06.184899 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff4196-5535-4d38-abce-72faf6043ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:06.185168 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff4196-5535-4d38-abce-72faf6043ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:06.206058 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8fff4196-5535-4d38-abce-72faf6043ab3 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:06.207272 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8fff4196-5535-4d38-abce-72faf6043ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:06.207600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8de234-8d40-48d3-b791-8b00811e80d4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:06.218215 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:06.250524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2cf4e9-67a1-4a88-9e9a-15adbe1f0402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:06.251272 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2cf4e9-67a1-4a88-9e9a-15adbe1f0402 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:06.251726 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2cf4e9-67a1-4a88-9e9a-15adbe1f0402 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:06.252008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2cf4e9-67a1-4a88-9e9a-15adbe1f0402 None admin] PATH_INFO: `/v3/projects/dac3a1a96e914a549133e73f03943618` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:06.253985 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2cf4e9-67a1-4a88-9e9a-15adbe1f0402 None admin] RBAC: Authorizing `identity:get_project(project_id=dac3a1a96e914a549133e73f03943618)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:06.255470 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2cf4e9-67a1-4a88-9e9a-15adbe1f0402 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:06.257279 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 222/440] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:14:06 2022] GET /identity/v3/projects/dac3a1a96e914a549133e73f03943618 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:06.263043 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2cf4e9-67a1-4a88-9e9a-15adbe1f0402 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:06.293002 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 219/441] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 09:14:06.294168 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff4196-5535-4d38-abce-72faf6043ab3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:06.371628 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:06.388549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2cf4e9-67a1-4a88-9e9a-15adbe1f0402 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:06.428537 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff4196-5535-4d38-abce-72faf6043ab3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:06.436064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446e485-fa35-45e2-93f9-56d41f781245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:06.436699 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446e485-fa35-45e2-93f9-56d41f781245 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:06.436979 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446e485-fa35-45e2-93f9-56d41f781245 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:06.437250 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0446e485-fa35-45e2-93f9-56d41f781245 None admin] PATH_INFO: `/v3/groups/8a53ec1153c1430ea8a4ac281c27da8b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:06.438379 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0446e485-fa35-45e2-93f9-56d41f781245 None admin] RBAC: Authorizing `identity:get_group(group_id=8a53ec1153c1430ea8a4ac281c27da8b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:06.439851 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0446e485-fa35-45e2-93f9-56d41f781245 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:06.441618 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 223/442] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Sep 2 09:14:06 2022] GET /identity/v3/groups/8a53ec1153c1430ea8a4ac281c27da8b => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:06.447677 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446e485-fa35-45e2-93f9-56d41f781245 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:06.474162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7996c36-23ca-4f17-be04-b44812144883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:06.475220 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7996c36-23ca-4f17-be04-b44812144883 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:06.475858 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7996c36-23ca-4f17-be04-b44812144883 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:06.476301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7996c36-23ca-4f17-be04-b44812144883 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:06.490793 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7996c36-23ca-4f17-be04-b44812144883 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:06.496313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-c7996c36-23ca-4f17-be04-b44812144883 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 09:14:06.496313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:06.496313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:06.496313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:06.496313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:06.496313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:06.496313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:06.496313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:06.496313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:06.496313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:06.496313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:06.496313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:06.496313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:06.496313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:06.496313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:06.496313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:06.497214 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:06.497214 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:06.497214 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:06.497214 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:06.497214 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:06.497214 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:06.497214 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:06.497214 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:06.497214 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:06.497214 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:06.497214 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:06.497214 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:06.497214 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:06.497214 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:06.497214 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:06.497214 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:06.497995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:06.497995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:06.497995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:06.497995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:06.497995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:06.497995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:06.497995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:06.497995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:06.497995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:06.497995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:06.497995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 09:14:06.497995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:06.502183 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 220/443] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Sep 2 09:14:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:06.503042 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:06.503456 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7996c36-23ca-4f17-be04-b44812144883 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:06.571451 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0446e485-fa35-45e2-93f9-56d41f781245 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:06.606873 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de10bf9-c273-44cb-98e6-7002f7bcc28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:06.607476 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de10bf9-c273-44cb-98e6-7002f7bcc28f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:06.607750 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de10bf9-c273-44cb-98e6-7002f7bcc28f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:06.608016 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de10bf9-c273-44cb-98e6-7002f7bcc28f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:06.608728 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de10bf9-c273-44cb-98e6-7002f7bcc28f None admin] RBAC: Adding query filter params (group.id=8a53ec1153c1430ea8a4ac281c27da8b, role.id=efe6bdd5112340b48b262ceafbc19c04, scope.project.id=dac3a1a96e914a549133e73f03943618) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:06.609032 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de10bf9-c273-44cb-98e6-7002f7bcc28f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:06.610462 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de10bf9-c273-44cb-98e6-7002f7bcc28f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:06.611203 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7996c36-23ca-4f17-be04-b44812144883 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:06.616059 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 224/444] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Sep 2 09:14:06 2022] GET /identity/v3/role_assignments?group.id=8a53ec1153c1430ea8a4ac281c27da8b&scope.project.id=dac3a1a96e914a549133e73f03943618&role.id=efe6bdd5112340b48b262ceafbc19c04 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:14:06.616939 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de10bf9-c273-44cb-98e6-7002f7bcc28f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:06.635895 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:06.655472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249f3db-6e5b-491e-adca-74ea2c0df0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:06.656176 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249f3db-6e5b-491e-adca-74ea2c0df0c1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:06.656378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249f3db-6e5b-491e-adca-74ea2c0df0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:06.656647 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249f3db-6e5b-491e-adca-74ea2c0df0c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:06.657330 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249f3db-6e5b-491e-adca-74ea2c0df0c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:06.657627 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249f3db-6e5b-491e-adca-74ea2c0df0c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:06.659278 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249f3db-6e5b-491e-adca-74ea2c0df0c1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:06.673471 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 221/445] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:14:06.674700 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249f3db-6e5b-491e-adca-74ea2c0df0c1 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:06.744931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de10bf9-c273-44cb-98e6-7002f7bcc28f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:06.752654 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:06.788757 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d45b22-d081-4f24-873d-811f2c6e2aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:06.789459 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d45b22-d081-4f24-873d-811f2c6e2aac None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:06.790278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d45b22-d081-4f24-873d-811f2c6e2aac None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:06.790552 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d45b22-d081-4f24-873d-811f2c6e2aac None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:06.800087 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249f3db-6e5b-491e-adca-74ea2c0df0c1 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:06.802562 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d45b22-d081-4f24-873d-811f2c6e2aac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:06.808710 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-06d45b22-d081-4f24-873d-811f2c6e2aac None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:14:06.808710 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:06.808710 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:06.808710 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:06.808710 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:06.808710 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:06.808710 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:06.808710 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:06.808710 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:06.808710 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:06.808710 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:06.808710 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:06.808710 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:06.808710 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:06.808710 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:06.808710 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:06.809592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:06.809592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:06.809592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:06.809592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:06.809592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:06.809592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:06.809592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:06.809592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:06.809592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:06.809592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:06.809592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:06.809592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:06.809592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:06.809592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:06.809592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:06.809592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:06.810372 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:06.810372 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:06.810372 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:06.810372 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:06.810372 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:06.810372 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:06.810372 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:06.810372 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:06.810372 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:06.810372 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:06.810372 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:14:06.810372 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:06.810372 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 225/446] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:14:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:06.815529 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d45b22-d081-4f24-873d-811f2c6e2aac None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:06.846521 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4563d1-c191-4faa-bcd2-4961c42089ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:06.847206 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4563d1-c191-4faa-bcd2-4961c42089ac None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:06.847484 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4563d1-c191-4faa-bcd2-4961c42089ac None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:06.847755 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4563d1-c191-4faa-bcd2-4961c42089ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:06.911730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:06.932626 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4563d1-c191-4faa-bcd2-4961c42089ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:06.933648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4563d1-c191-4faa-bcd2-4961c42089ac None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:06.942134 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d45b22-d081-4f24-873d-811f2c6e2aac None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:06.982046 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9fb06-7277-493f-9f85-ba0cc65e4b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:06.982660 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9fb06-7277-493f-9f85-ba0cc65e4b5c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:06.982980 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9fb06-7277-493f-9f85-ba0cc65e4b5c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:06.983256 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9fb06-7277-493f-9f85-ba0cc65e4b5c None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:06.983924 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9fb06-7277-493f-9f85-ba0cc65e4b5c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:06.984230 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9fb06-7277-493f-9f85-ba0cc65e4b5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:06.985605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9fb06-7277-493f-9f85-ba0cc65e4b5c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:06.993767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 226/447] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:14:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:14:06.999233 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9fb06-7277-493f-9f85-ba0cc65e4b5c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:07.026955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:07.045974 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 222/448] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:14:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:14:07.051635 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4563d1-c191-4faa-bcd2-4961c42089ac None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:07.099049 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9fb06-7277-493f-9f85-ba0cc65e4b5c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:07.130859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:07.131040 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6292fd-64f8-4824-9850-dfe521037468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:07.131865 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6292fd-64f8-4824-9850-dfe521037468 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:07.132150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6292fd-64f8-4824-9850-dfe521037468 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:07.132420 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6292fd-64f8-4824-9850-dfe521037468 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:07.150311 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4563d1-c191-4faa-bcd2-4961c42089ac None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:07.183529 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154c1e1-142d-4a80-8f3b-ce3d15ad0e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:07.184147 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154c1e1-142d-4a80-8f3b-ce3d15ad0e55 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:07.184427 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154c1e1-142d-4a80-8f3b-ce3d15ad0e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:07.184698 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154c1e1-142d-4a80-8f3b-ce3d15ad0e55 None admin] PATH_INFO: `/v3/projects/dac3a1a96e914a549133e73f03943618` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:07.185823 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154c1e1-142d-4a80-8f3b-ce3d15ad0e55 None admin] RBAC: Authorizing `identity:get_project(project_id=dac3a1a96e914a549133e73f03943618)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:07.187287 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154c1e1-142d-4a80-8f3b-ce3d15ad0e55 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:07.188974 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 223/449] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:14:07 2022] GET /identity/v3/projects/dac3a1a96e914a549133e73f03943618 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:14:07.194370 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154c1e1-142d-4a80-8f3b-ce3d15ad0e55 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:07.201200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6292fd-64f8-4824-9850-dfe521037468 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:07.202735 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6292fd-64f8-4824-9850-dfe521037468 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:07.230644 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:07.295538 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154c1e1-142d-4a80-8f3b-ce3d15ad0e55 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:07.303658 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 227/450] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:14:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 02 09:14:07.309723 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6292fd-64f8-4824-9850-dfe521037468 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:07.332289 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5794e-ce3b-4749-8e96-36ce99b74c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:07.332915 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5794e-ce3b-4749-8e96-36ce99b74c99 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:07.333195 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5794e-ce3b-4749-8e96-36ce99b74c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:07.333477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5794e-ce3b-4749-8e96-36ce99b74c99 None admin] PATH_INFO: `/v3/groups/8a53ec1153c1430ea8a4ac281c27da8b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:07.334592 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb5794e-ce3b-4749-8e96-36ce99b74c99 None admin] RBAC: Authorizing `identity:get_group(group_id=8a53ec1153c1430ea8a4ac281c27da8b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:07.336071 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb5794e-ce3b-4749-8e96-36ce99b74c99 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:07.337663 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 224/451] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Sep 2 09:14:07 2022] GET /identity/v3/groups/8a53ec1153c1430ea8a4ac281c27da8b => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:14:07.343273 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5794e-ce3b-4749-8e96-36ce99b74c99 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:07.360977 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:07.406275 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6292fd-64f8-4824-9850-dfe521037468 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:07.432664 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:07.440450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abdf3b-dfb2-429b-aaba-186d45caf41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:07.441077 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abdf3b-dfb2-429b-aaba-186d45caf41f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:07.441353 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abdf3b-dfb2-429b-aaba-186d45caf41f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:07.441624 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abdf3b-dfb2-429b-aaba-186d45caf41f None admin] PATH_INFO: `/v3/projects/ae5820815add48cd913951058cc9251c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:07.442880 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20abdf3b-dfb2-429b-aaba-186d45caf41f None admin] RBAC: Authorizing `identity:get_project(project_id=ae5820815add48cd913951058cc9251c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:07.444268 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20abdf3b-dfb2-429b-aaba-186d45caf41f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:07.444418 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5794e-ce3b-4749-8e96-36ce99b74c99 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:07.445927 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 228/452] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:14:07 2022] GET /identity/v3/projects/ae5820815add48cd913951058cc9251c => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:14:07.452199 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abdf3b-dfb2-429b-aaba-186d45caf41f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:07.479705 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3999c-c4f8-44d9-acc0-116efaa10a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:07.480311 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3999c-c4f8-44d9-acc0-116efaa10a69 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:07.480597 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3999c-c4f8-44d9-acc0-116efaa10a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:07.480876 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d3999c-c4f8-44d9-acc0-116efaa10a69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:07.481601 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3999c-c4f8-44d9-acc0-116efaa10a69 None admin] RBAC: Adding query filter params (group.id=8a53ec1153c1430ea8a4ac281c27da8b, role.id=f2fe9fb31024471e9e2bea2b12715de6, scope.project.id=dac3a1a96e914a549133e73f03943618) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:07.481907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3999c-c4f8-44d9-acc0-116efaa10a69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:07.483395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d3999c-c4f8-44d9-acc0-116efaa10a69 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:07.489048 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 225/453] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Sep 2 09:14:07 2022] GET /identity/v3/role_assignments?group.id=8a53ec1153c1430ea8a4ac281c27da8b&scope.project.id=dac3a1a96e914a549133e73f03943618&role.id=f2fe9fb31024471e9e2bea2b12715de6 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:07.557306 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abdf3b-dfb2-429b-aaba-186d45caf41f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:07.592818 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077657d-7047-454a-b315-32e3b642a293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:07.593459 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077657d-7047-454a-b315-32e3b642a293 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:07.593737 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077657d-7047-454a-b315-32e3b642a293 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:07.594008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077657d-7047-454a-b315-32e3b642a293 None admin] PATH_INFO: `/v3/groups/b14338530049456aa7d40c06b40af01d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:07.595198 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c077657d-7047-454a-b315-32e3b642a293 None admin] RBAC: Authorizing `identity:get_group(group_id=b14338530049456aa7d40c06b40af01d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:07.596641 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c077657d-7047-454a-b315-32e3b642a293 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:07.598260 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 229/454] 10.222.0.57 () {56 vars in 1180 bytes} [Fri Sep 2 09:14:07 2022] GET /identity/v3/groups/b14338530049456aa7d40c06b40af01d => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:14:07.604878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3999c-c4f8-44d9-acc0-116efaa10a69 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:07.693355 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:07.714056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d3999c-c4f8-44d9-acc0-116efaa10a69 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:07.747970 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98525f5f-3a5b-4100-83eb-514241ff4f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:07.748991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98525f5f-3a5b-4100-83eb-514241ff4f50 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:07.749490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98525f5f-3a5b-4100-83eb-514241ff4f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:07.749981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98525f5f-3a5b-4100-83eb-514241ff4f50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:07.751525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98525f5f-3a5b-4100-83eb-514241ff4f50 None admin] RBAC: Adding query filter params (group.id=b14338530049456aa7d40c06b40af01d, role.id=f886b8e8889549e896a8fb3426cce9e1, scope.project.id=ae5820815add48cd913951058cc9251c) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:07.751933 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98525f5f-3a5b-4100-83eb-514241ff4f50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:07.754630 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98525f5f-3a5b-4100-83eb-514241ff4f50 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:07.762093 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 226/455] 10.222.0.57 () {56 vars in 1376 bytes} [Fri Sep 2 09:14:07 2022] GET /identity/v3/role_assignments?group.id=b14338530049456aa7d40c06b40af01d&scope.project.id=ae5820815add48cd913951058cc9251c&role.id=f886b8e8889549e896a8fb3426cce9e1 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:10.024928 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111088d-ccbe-40f2-9f83-5b8431aa78d3 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:10.025951 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111088d-ccbe-40f2-9f83-5b8431aa78d3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:10.026433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111088d-ccbe-40f2-9f83-5b8431aa78d3 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:10.028026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 230/456] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:10.048749 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177223b-1a34-44c9-8936-75d3c404e3f4 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:10.049650 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177223b-1a34-44c9-8936-75d3c404e3f4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:10.050111 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b177223b-1a34-44c9-8936-75d3c404e3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:10.079752 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b177223b-1a34-44c9-8936-75d3c404e3f4 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:10.081301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b177223b-1a34-44c9-8936-75d3c404e3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:10.160390 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 227/457] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:10.289134 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca51d1b-512f-4af7-bf84-825fcffaea1c None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:10.290309 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca51d1b-512f-4af7-bf84-825fcffaea1c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:10.290828 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca51d1b-512f-4af7-bf84-825fcffaea1c None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:10.292674 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 231/458] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:10.297632 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b668162-f21e-4802-a2e9-8671da3a82aa None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:10.297925 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b668162-f21e-4802-a2e9-8671da3a82aa None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:10.298207 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b668162-f21e-4802-a2e9-8671da3a82aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:10.319724 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4b668162-f21e-4802-a2e9-8671da3a82aa None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:10.321490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4b668162-f21e-4802-a2e9-8671da3a82aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:10.407400 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 228/459] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:10.408339 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda82ec-c87b-4a4a-947b-63dfd1c371ef None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:10.408628 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda82ec-c87b-4a4a-947b-63dfd1c371ef None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:10.408904 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda82ec-c87b-4a4a-947b-63dfd1c371ef None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:10.410227 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 232/460] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:10.415352 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867d51d-b4fa-4f1b-8c45-b764d0f50721 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:10.415593 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867d51d-b4fa-4f1b-8c45-b764d0f50721 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:10.415876 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6867d51d-b4fa-4f1b-8c45-b764d0f50721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:10.428212 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:10.435728 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6867d51d-b4fa-4f1b-8c45-b764d0f50721 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:10.437076 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6867d51d-b4fa-4f1b-8c45-b764d0f50721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:10.512221 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda82ec-c87b-4a4a-947b-63dfd1c371ef None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:10.520800 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 229/461] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:10.609526 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:10.662884 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda82ec-c87b-4a4a-947b-63dfd1c371ef None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:10.708294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39938dad-46ed-4e78-a7eb-8f1cf8d4c247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:10.708841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39938dad-46ed-4e78-a7eb-8f1cf8d4c247 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:10.709116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39938dad-46ed-4e78-a7eb-8f1cf8d4c247 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:10.709383 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39938dad-46ed-4e78-a7eb-8f1cf8d4c247 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:10.714727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867d51d-b4fa-4f1b-8c45-b764d0f50721 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:10.719993 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39938dad-46ed-4e78-a7eb-8f1cf8d4c247 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:10.721443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39938dad-46ed-4e78-a7eb-8f1cf8d4c247 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:10.746292 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-39938dad-46ed-4e78-a7eb-8f1cf8d4c247 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:10.746292 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:10.746292 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:10.746292 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:10.746292 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:10.746292 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:10.746292 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:10.746292 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:10.746292 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:10.746292 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:10.746292 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:10.746292 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:10.746292 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:10.746292 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:10.746292 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:10.746292 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:10.747278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:10.747278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:10.747278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:10.747278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:10.747278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:10.747278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:10.747278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:10.747278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:10.747278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:10.747278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:10.747278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:10.747278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:10.747278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:10.747278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:10.747278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:10.747278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:10.747278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:10.748117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:10.748117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:10.748117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:10.748841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 233/462] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:14:10.757099 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39938dad-46ed-4e78-a7eb-8f1cf8d4c247 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:10.912553 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6867d51d-b4fa-4f1b-8c45-b764d0f50721 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:10.916256 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39938dad-46ed-4e78-a7eb-8f1cf8d4c247 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:10.933744 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:10.953131 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0af2e8-b098-4630-9747-4d66c1ecf274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:10.953764 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0af2e8-b098-4630-9747-4d66c1ecf274 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:10.953863 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a39d3-9756-4e7c-ae9c-2e618b85394b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:10.954079 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0af2e8-b098-4630-9747-4d66c1ecf274 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:10.954304 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0af2e8-b098-4630-9747-4d66c1ecf274 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:10.954614 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a39d3-9756-4e7c-ae9c-2e618b85394b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:10.954991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a39d3-9756-4e7c-ae9c-2e618b85394b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:10.955080 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0af2e8-b098-4630-9747-4d66c1ecf274 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:10.955361 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a39d3-9756-4e7c-ae9c-2e618b85394b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:10.955462 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0af2e8-b098-4630-9747-4d66c1ecf274 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:10.956813 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0af2e8-b098-4630-9747-4d66c1ecf274 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:10.958844 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:10.968858 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8a39d3-9756-4e7c-ae9c-2e618b85394b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:10.971460 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8a39d3-9756-4e7c-ae9c-2e618b85394b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:10.974189 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 234/463] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:10.975722 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8cade7-8546-40be-ab8c-924f837650bf None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:10.976246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8cade7-8546-40be-ab8c-924f837650bf None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:10.976651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8cade7-8546-40be-ab8c-924f837650bf None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:10.978027 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 235/464] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:10.979024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8cade7-8546-40be-ab8c-924f837650bf None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:10.987780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-fc8a39d3-9756-4e7c-ae9c-2e618b85394b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:10.987780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:10.987780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:10.987780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:10.987780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:10.987780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:10.987780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:10.987780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:10.987780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:10.987780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:10.987780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:10.987780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:10.987780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:10.987780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:10.987780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:10.987780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:10.988726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:10.988726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:10.988726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:10.988726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:10.988726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:10.988726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:10.988726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:10.988726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:10.988726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:10.988726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:10.988726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:10.988726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:10.988726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:10.988726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:10.988726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:10.988726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:10.988726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:10.989718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:10.989718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:10.989718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:10.989718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 230/465] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:10.994796 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a39d3-9756-4e7c-ae9c-2e618b85394b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:11.090177 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8cade7-8546-40be-ab8c-924f837650bf None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:11.097233 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a39d3-9756-4e7c-ae9c-2e618b85394b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:11.126086 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f290b-737a-40d2-b228-d4b5c52a3c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:11.126738 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f290b-737a-40d2-b228-d4b5c52a3c7a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:11.127019 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f290b-737a-40d2-b228-d4b5c52a3c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:11.127285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f290b-737a-40d2-b228-d4b5c52a3c7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:11.132670 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29204fb9-d2f3-47b2-ba43-b5160220f400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:11.133818 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29204fb9-d2f3-47b2-ba43-b5160220f400 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:11.134098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29204fb9-d2f3-47b2-ba43-b5160220f400 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:11.134811 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29204fb9-d2f3-47b2-ba43-b5160220f400 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:11.135456 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29204fb9-d2f3-47b2-ba43-b5160220f400 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:11.137305 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29204fb9-d2f3-47b2-ba43-b5160220f400 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:11.137777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122f290b-737a-40d2-b228-d4b5c52a3c7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:11.139313 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122f290b-737a-40d2-b228-d4b5c52a3c7a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:11.154673 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-122f290b-737a-40d2-b228-d4b5c52a3c7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:11.154673 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:11.154673 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:11.154673 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:11.154673 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:11.154673 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:11.154673 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:11.154673 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:11.154673 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:11.154673 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:11.154673 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:11.154673 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:11.154673 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:11.154673 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:11.154673 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:11.154673 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:11.155648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:11.155648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:11.155648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:11.155648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:11.155648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:11.155648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:11.155648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:11.155648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:11.155648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:11.155648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:11.155648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:11.155648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:11.155648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:11.155648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:11.155648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:11.155648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:11.155648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:11.156471 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:11.156471 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:11.156471 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:11.156471 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 236/466] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:11.157163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a96bcf-312a-42e8-bad8-46884c06d8d1 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:11.157393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a96bcf-312a-42e8-bad8-46884c06d8d1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:11.157726 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a96bcf-312a-42e8-bad8-46884c06d8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:11.175256 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d1a96bcf-312a-42e8-bad8-46884c06d8d1 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:11.176428 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d1a96bcf-312a-42e8-bad8-46884c06d8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:11.199864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:11.244144 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 237/467] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:11.244907 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a96bcf-312a-42e8-bad8-46884c06d8d1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:11.259883 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 231/468] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Sep 2 09:14:10 2022] POST /identity/v3/users => generated 246 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:14:11.259883 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29204fb9-d2f3-47b2-ba43-b5160220f400 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:11.349783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:11.440721 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a96bcf-312a-42e8-bad8-46884c06d8d1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:11.447477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29204fb9-d2f3-47b2-ba43-b5160220f400 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:11.479273 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224bd6f-5318-402f-aa9b-9ed6a878a257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:11.480302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224bd6f-5318-402f-aa9b-9ed6a878a257 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:11.480863 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224bd6f-5318-402f-aa9b-9ed6a878a257 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:11.481307 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224bd6f-5318-402f-aa9b-9ed6a878a257 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:11.482122 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6224bd6f-5318-402f-aa9b-9ed6a878a257 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:11.482610 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6224bd6f-5318-402f-aa9b-9ed6a878a257 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:11.484466 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6224bd6f-5318-402f-aa9b-9ed6a878a257 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:11.489424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c8ca2-c1ee-4b5c-b194-6f1b67a3edfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:11.490118 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c8ca2-c1ee-4b5c-b194-6f1b67a3edfd None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:11.490489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c8ca2-c1ee-4b5c-b194-6f1b67a3edfd None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:11.490879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c8ca2-c1ee-4b5c-b194-6f1b67a3edfd None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:11.491620 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c8ca2-c1ee-4b5c-b194-6f1b67a3edfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:11.491995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c8ca2-c1ee-4b5c-b194-6f1b67a3edfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:11.494234 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c8ca2-c1ee-4b5c-b194-6f1b67a3edfd None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:11.499171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 238/469] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:11.504892 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224bd6f-5318-402f-aa9b-9ed6a878a257 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:11.507551 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 232/470] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:11.515730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c8ca2-c1ee-4b5c-b194-6f1b67a3edfd None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:11.530968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:11.629374 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c8ca2-c1ee-4b5c-b194-6f1b67a3edfd None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:11.635129 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224bd6f-5318-402f-aa9b-9ed6a878a257 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:11.638268 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:11.676224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e150c-d6cf-40b1-ace6-06b68042f4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:11.676789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ebfa8-528b-4b66-bf13-d29c8b32d516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:11.677338 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e150c-d6cf-40b1-ace6-06b68042f4d5 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:11.678046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e150c-d6cf-40b1-ace6-06b68042f4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:11.678178 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ebfa8-528b-4b66-bf13-d29c8b32d516 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:11.678486 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e150c-d6cf-40b1-ace6-06b68042f4d5 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:11.678780 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ebfa8-528b-4b66-bf13-d29c8b32d516 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:11.679207 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ebfa8-528b-4b66-bf13-d29c8b32d516 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:11.679460 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e150c-d6cf-40b1-ace6-06b68042f4d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:11.680174 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47ebfa8-528b-4b66-bf13-d29c8b32d516 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:11.681108 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e150c-d6cf-40b1-ace6-06b68042f4d5 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:11.681902 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47ebfa8-528b-4b66-bf13-d29c8b32d516 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:11.780326 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 233/471] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Sep 2 09:14:11 2022] POST /identity/v3/users => generated 248 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:14:11.786006 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e150c-d6cf-40b1-ace6-06b68042f4d5 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:11.816910 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 239/472] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Sep 2 09:14:11 2022] POST /identity/v3/users => generated 248 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:14:11.828307 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b3106-1383-4118-aacc-6131934c3853 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:11.828723 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b3106-1383-4118-aacc-6131934c3853 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:11.828901 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b3106-1383-4118-aacc-6131934c3853 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:11.830446 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 240/473] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:11.836122 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450be61-64fc-4c3b-b48a-cb194a004738 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:11.836449 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450be61-64fc-4c3b-b48a-cb194a004738 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:11.836675 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450be61-64fc-4c3b-b48a-cb194a004738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:11.871955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a450be61-64fc-4c3b-b48a-cb194a004738 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:11.873202 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a450be61-64fc-4c3b-b48a-cb194a004738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:11.922321 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e150c-d6cf-40b1-ace6-06b68042f4d5 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:11.983622 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9815046-602d-40a5-8424-a5cf902ade38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:11.984302 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9815046-602d-40a5-8424-a5cf902ade38 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:11.984588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9815046-602d-40a5-8424-a5cf902ade38 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:11.984867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9815046-602d-40a5-8424-a5cf902ade38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:12.011796 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9815046-602d-40a5-8424-a5cf902ade38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:12.013289 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9815046-602d-40a5-8424-a5cf902ade38 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:12.018865 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 241/474] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 09:14:12.030395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d9815046-602d-40a5-8424-a5cf902ade38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:12.030395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:12.030395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:12.030395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:12.030395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:12.030395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:12.030395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:12.030395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:12.030395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:12.030395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:12.030395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:12.030395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:12.030395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:12.030395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:12.030395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:12.030395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:12.031333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:12.031333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:12.031333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:12.031333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:12.031333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:12.031333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:12.031333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:12.031333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:12.031333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:12.031333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:12.031333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:12.031333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:12.031333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:12.031333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:12.031333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:12.031333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:12.031333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:12.032191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:12.032191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:12.032191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:12.032783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 234/475] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:12.038857 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450be61-64fc-4c3b-b48a-cb194a004738 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:12.048718 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:12.149406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450be61-64fc-4c3b-b48a-cb194a004738 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:12.186086 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba25945-65a1-4831-b51c-a6815c186f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:12.186691 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba25945-65a1-4831-b51c-a6815c186f62 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:12.187000 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba25945-65a1-4831-b51c-a6815c186f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:12.187275 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba25945-65a1-4831-b51c-a6815c186f62 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:12.187926 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba25945-65a1-4831-b51c-a6815c186f62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:12.188231 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba25945-65a1-4831-b51c-a6815c186f62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:12.189588 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba25945-65a1-4831-b51c-a6815c186f62 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:12.202601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 242/476] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:12.208175 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9815046-602d-40a5-8424-a5cf902ade38 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:12.230930 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:12.328380 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9815046-602d-40a5-8424-a5cf902ade38 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:12.331178 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba25945-65a1-4831-b51c-a6815c186f62 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:12.375837 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fc99f-ec3a-492a-916c-6f98ca6b483b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:12.379865 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fc99f-ec3a-492a-916c-6f98ca6b483b None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:12.380362 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fc99f-ec3a-492a-916c-6f98ca6b483b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:12.380924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26fc99f-ec3a-492a-916c-6f98ca6b483b None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:12.381766 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26fc99f-ec3a-492a-916c-6f98ca6b483b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:12.383947 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26fc99f-ec3a-492a-916c-6f98ca6b483b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:12.432421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:12.473349 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba25945-65a1-4831-b51c-a6815c186f62 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:12.502312 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 235/477] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Sep 2 09:14:12 2022] POST /identity/v3/users => generated 249 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:14:12.518290 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471af599-1e27-4e16-8283-b800e39b2899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:12.519295 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471af599-1e27-4e16-8283-b800e39b2899 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:12.524409 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471af599-1e27-4e16-8283-b800e39b2899 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:12.524409 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471af599-1e27-4e16-8283-b800e39b2899 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:12.524409 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471af599-1e27-4e16-8283-b800e39b2899 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:12.524409 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471af599-1e27-4e16-8283-b800e39b2899 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:12.562137 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.sql.core [None req-471af599-1e27-4e16-8283-b800e39b2899 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 02 09:14:12.562137 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 02 09:14:12.562137 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [parameters: {'id': '1cd7775ec4cc456e9a2ce1ae61064ea8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 02 09:14:12.562137 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81611) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 02 09:14:12.580008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-471af599-1e27-4e16-8283-b800e39b2899 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 02 09:14:12.580008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:12.580008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 02 09:14:12.580008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 02 09:14:12.580008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 02 09:14:12.580008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 02 09:14:12.580008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 02 09:14:12.580008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result = self._query(query) Sep 02 09:14:12.580008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 02 09:14:12.580008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application conn.query(q) Sep 02 09:14:12.580008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 02 09:14:12.580008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 02 09:14:12.580008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 02 09:14:12.580008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result.read() Sep 02 09:14:12.580008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 02 09:14:12.580955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 02 09:14:12.580955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 02 09:14:12.580955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 02 09:14:12.580955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 02 09:14:12.580955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 02 09:14:12.580955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 02 09:14:12.580955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 02 09:14:12.580955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 02 09:14:12.580955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:12.580955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 02 09:14:12.580955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:12.580955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:12.580955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 02 09:14:12.580955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 02 09:14:12.580955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 02 09:14:12.580955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return ref.to_dict() Sep 02 09:14:12.580955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 02 09:14:12.580955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application next(self.gen) Sep 02 09:14:12.581860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 02 09:14:12.581860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application yield resource Sep 02 09:14:12.581860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 02 09:14:12.581860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application next(self.gen) Sep 02 09:14:12.581860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 02 09:14:12.581860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.session.rollback() Sep 02 09:14:12.581860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 02 09:14:12.581860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.force_reraise() Sep 02 09:14:12.581860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 02 09:14:12.581860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise self.value Sep 02 09:14:12.581860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 02 09:14:12.581860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 02 09:14:12.581860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 02 09:14:12.581860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application session.commit() Sep 02 09:14:12.581860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 02 09:14:12.581860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 02 09:14:12.581860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 02 09:14:12.582686 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._prepare_impl() Sep 02 09:14:12.582686 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 02 09:14:12.582686 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.session.flush() Sep 02 09:14:12.582686 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 02 09:14:12.582686 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._flush(objects) Sep 02 09:14:12.582686 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 02 09:14:12.582686 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 02 09:14:12.582686 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 02 09:14:12.582686 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application compat.raise_( Sep 02 09:14:12.582686 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 02 09:14:12.582686 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception Sep 02 09:14:12.582686 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 02 09:14:12.582686 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application flush_context.execute() Sep 02 09:14:12.582686 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 02 09:14:12.582686 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rec.execute(self) Sep 02 09:14:12.582686 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 02 09:14:12.582686 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 02 09:14:12.583614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 02 09:14:12.583614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 02 09:14:12.583614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 02 09:14:12.583614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 02 09:14:12.583614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 02 09:14:12.583614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 02 09:14:12.583614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 02 09:14:12.583614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 02 09:14:12.583614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 02 09:14:12.583614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 02 09:14:12.583614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 02 09:14:12.583614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 02 09:14:12.583614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 02 09:14:12.583614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 02 09:14:12.583614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 02 09:14:12.583614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception Sep 02 09:14:12.584439 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 02 09:14:12.584439 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 02 09:14:12.584439 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 02 09:14:12.584439 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 02 09:14:12.584439 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 02 09:14:12.584439 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result = self._query(query) Sep 02 09:14:12.584439 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 02 09:14:12.584439 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application conn.query(q) Sep 02 09:14:12.584439 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 02 09:14:12.584439 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 02 09:14:12.584439 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 02 09:14:12.584439 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result.read() Sep 02 09:14:12.584439 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 02 09:14:12.584439 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 02 09:14:12.584439 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 02 09:14:12.584439 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 02 09:14:12.587166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 02 09:14:12.587166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 02 09:14:12.587166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 02 09:14:12.587166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 02 09:14:12.587166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 02 09:14:12.587166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 02 09:14:12.587166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [parameters: {'id': '1cd7775ec4cc456e9a2ce1ae61064ea8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 02 09:14:12.587166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 02 09:14:12.587166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:12.587166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:14:12.587166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:12.587166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:12.587166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:12.587166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:12.587166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:12.587166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:12.589270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:12.589270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:12.589270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:12.589270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:12.589270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:12.589270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:12.589270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 02 09:14:12.589270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 02 09:14:12.589270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:12.589270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:12.589270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 02 09:14:12.589270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 02 09:14:12.589270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 02 09:14:12.589270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 02 09:14:12.589270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 02 09:14:12.589270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:12.591390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 243/478] 10.222.0.57 () {60 vars in 1131 bytes} [Fri Sep 2 09:14:12 2022] POST /identity/v3/roles => generated 144 bytes in 251 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:14:12.591539 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fc99f-ec3a-492a-916c-6f98ca6b483b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:12.693241 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:12.729392 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26fc99f-ec3a-492a-916c-6f98ca6b483b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:12.765822 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938ec9de-3aa5-432a-8cd0-420a0299e2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:12.766834 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ec9de-3aa5-432a-8cd0-420a0299e2b3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:12.767315 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ec9de-3aa5-432a-8cd0-420a0299e2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:12.767785 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938ec9de-3aa5-432a-8cd0-420a0299e2b3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:12.780452 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938ec9de-3aa5-432a-8cd0-420a0299e2b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:12.786852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-938ec9de-3aa5-432a-8cd0-420a0299e2b3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 09:14:12.786852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:12.786852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:12.786852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:12.786852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:12.786852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:12.786852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:12.786852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:12.786852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:12.786852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:12.786852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:12.786852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:12.786852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:12.786852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:12.786852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:12.786852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:12.787789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:12.787789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:12.787789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:12.787789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:12.787789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:12.787789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:12.787789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:12.787789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:12.787789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:12.787789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:12.787789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:12.787789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:12.787789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:12.787789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:12.787789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:12.787789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:12.788571 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:12.788571 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:12.788571 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:12.788571 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:12.788571 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:12.788571 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:12.788571 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:12.788571 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:12.788571 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:12.788571 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:12.788571 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 02 09:14:12.788571 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:12.791105 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 236/479] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Sep 2 09:14:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:12.800171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471af599-1e27-4e16-8283-b800e39b2899 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:12.910423 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471af599-1e27-4e16-8283-b800e39b2899 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:12.937874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:12.949672 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e990def-3c99-4fa4-9bf3-5dc503782e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:12.950450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e990def-3c99-4fa4-9bf3-5dc503782e69 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:12.950860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e990def-3c99-4fa4-9bf3-5dc503782e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:12.951188 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e990def-3c99-4fa4-9bf3-5dc503782e69 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:12.953902 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e990def-3c99-4fa4-9bf3-5dc503782e69 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:12.953902 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e990def-3c99-4fa4-9bf3-5dc503782e69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:12.953902 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e990def-3c99-4fa4-9bf3-5dc503782e69 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:12.964058 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 244/480] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:13.574537 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598569c-4130-4e57-a27c-700216254163 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:13.575124 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598569c-4130-4e57-a27c-700216254163 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:13.575442 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598569c-4130-4e57-a27c-700216254163 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:13.577175 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 237/481] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:13.582424 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca754d7-ad71-4646-83e7-20b1f666099a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:13.582861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca754d7-ad71-4646-83e7-20b1f666099a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:13.583061 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca754d7-ad71-4646-83e7-20b1f666099a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:13.605333 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-eca754d7-ad71-4646-83e7-20b1f666099a None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:13.606535 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-eca754d7-ad71-4646-83e7-20b1f666099a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:13.696930 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 245/482] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:14.059146 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6598569c-4130-4e57-a27c-700216254163 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:14.182381 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f4a9e-4bc0-4ddd-9652-d0a23b8c7e54 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:14.182744 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f4a9e-4bc0-4ddd-9652-d0a23b8c7e54 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:14.182935 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f4a9e-4bc0-4ddd-9652-d0a23b8c7e54 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:14.184477 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 246/483] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:14.189283 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ed458-7bea-42b8-b79f-28ca2b737284 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:14.189568 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ed458-7bea-42b8-b79f-28ca2b737284 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:14.189817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ed458-7bea-42b8-b79f-28ca2b737284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:14.199683 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6598569c-4130-4e57-a27c-700216254163 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:14.211713 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-338ed458-7bea-42b8-b79f-28ca2b737284 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:14.212918 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-338ed458-7bea-42b8-b79f-28ca2b737284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:14.253693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:14.268777 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6660e3-b09a-4bb9-99bb-861b0a6043bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:14.269762 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6660e3-b09a-4bb9-99bb-861b0a6043bf None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:14.270236 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6660e3-b09a-4bb9-99bb-861b0a6043bf None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:14.270698 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6660e3-b09a-4bb9-99bb-861b0a6043bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:14.287151 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 247/484] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:14.298423 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6660e3-b09a-4bb9-99bb-861b0a6043bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:14.303215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-8c6660e3-b09a-4bb9-99bb-861b0a6043bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:14.303215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:14.303215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:14.303215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:14.303215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:14.303215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:14.303215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:14.303215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:14.303215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:14.303215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:14.303215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:14.303215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:14.303215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:14.303215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:14.303215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:14.303215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:14.304284 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:14.304284 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:14.304284 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:14.304284 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:14.304284 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:14.304284 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:14.304284 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:14.304284 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:14.304284 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:14.304284 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:14.304284 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:14.304284 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:14.304284 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:14.304284 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:14.304284 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:14.304284 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:14.305308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:14.305308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:14.305308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:14.305308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:14.305308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:14.305308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:14.305308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:14.305308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:14.305308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:14.305308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:14.305308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:14.305308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:14.305308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 238/485] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Sep 2 09:14:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:14.310326 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ed458-7bea-42b8-b79f-28ca2b737284 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:14.368896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf927b-3244-4a90-804b-2290fbc1c45f None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:14.369522 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf927b-3244-4a90-804b-2290fbc1c45f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:14.369991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf927b-3244-4a90-804b-2290fbc1c45f None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:14.371564 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 239/486] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:14.376712 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384768a7-d749-448e-bbf8-c210b3b63769 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:14.376993 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384768a7-d749-448e-bbf8-c210b3b63769 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:14.377264 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384768a7-d749-448e-bbf8-c210b3b63769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:14.382277 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:14.419701 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-384768a7-d749-448e-bbf8-c210b3b63769 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:14.420901 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-384768a7-d749-448e-bbf8-c210b3b63769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:14.435114 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ed458-7bea-42b8-b79f-28ca2b737284 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:14.471270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5576cd7-a643-4a55-b436-a7d04affb06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:14.471962 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5576cd7-a643-4a55-b436-a7d04affb06e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:14.472354 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5576cd7-a643-4a55-b436-a7d04affb06e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:14.472641 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5576cd7-a643-4a55-b436-a7d04affb06e None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:14.473369 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5576cd7-a643-4a55-b436-a7d04affb06e None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:14.473682 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5576cd7-a643-4a55-b436-a7d04affb06e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:14.475070 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5576cd7-a643-4a55-b436-a7d04affb06e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:14.483664 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 248/487] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Sep 2 09:14:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:14.489345 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5576cd7-a643-4a55-b436-a7d04affb06e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:14.555943 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 240/488] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:14.633577 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5576cd7-a643-4a55-b436-a7d04affb06e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:14.647731 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:14.675267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c77040e-53b0-43ea-9f1d-f237fb7d03f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:14.677457 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77040e-53b0-43ea-9f1d-f237fb7d03f1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:14.678016 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77040e-53b0-43ea-9f1d-f237fb7d03f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:14.678500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c77040e-53b0-43ea-9f1d-f237fb7d03f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:14.682920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384768a7-d749-448e-bbf8-c210b3b63769 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:14.718633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c77040e-53b0-43ea-9f1d-f237fb7d03f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:14.720457 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c77040e-53b0-43ea-9f1d-f237fb7d03f1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:14.749707 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4c77040e-53b0-43ea-9f1d-f237fb7d03f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:14.749707 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:14.749707 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:14.749707 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:14.749707 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:14.749707 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:14.749707 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:14.749707 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:14.749707 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:14.749707 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:14.749707 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:14.749707 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:14.749707 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:14.749707 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:14.749707 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:14.749707 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:14.750669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:14.750669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:14.750669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:14.750669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:14.750669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:14.750669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:14.750669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:14.750669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:14.750669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:14.750669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:14.750669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:14.750669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:14.750669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:14.750669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:14.750669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:14.750669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:14.750669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:14.751587 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:14.751587 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:14.751587 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:14.754142 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 249/489] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:14.760200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c77040e-53b0-43ea-9f1d-f237fb7d03f1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:14.832435 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384768a7-d749-448e-bbf8-c210b3b63769 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:14.871208 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:14.879936 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3b1ef-5c13-4370-bac7-9b011d5f8dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:14.881300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3b1ef-5c13-4370-bac7-9b011d5f8dfe None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:14.882089 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3b1ef-5c13-4370-bac7-9b011d5f8dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:14.882583 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3b1ef-5c13-4370-bac7-9b011d5f8dfe None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:14.898017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a3b1ef-5c13-4370-bac7-9b011d5f8dfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:14.904475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d5a3b1ef-5c13-4370-bac7-9b011d5f8dfe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:14.904475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:14.904475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:14.904475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:14.904475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:14.904475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:14.904475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:14.904475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:14.904475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:14.904475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:14.904475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:14.904475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:14.904475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:14.904475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:14.904475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:14.904475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:14.905378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:14.905378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:14.905378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:14.905378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:14.905378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:14.905378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:14.905378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:14.905378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:14.905378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:14.905378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:14.905378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:14.905378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:14.905378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:14.905378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:14.905378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:14.905378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:14.906188 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:14.906188 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:14.906188 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:14.906188 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:14.906188 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:14.906188 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:14.906188 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:14.906188 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:14.906188 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:14.906188 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:14.906188 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:14.906188 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:14.914477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 241/490] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Sep 2 09:14:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:14:14.916347 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3b1ef-5c13-4370-bac7-9b011d5f8dfe None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:14.950140 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c77040e-53b0-43ea-9f1d-f237fb7d03f1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:14.991464 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9de3d-5a99-4d03-bf95-40b7aece69bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:14.992004 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9de3d-5a99-4d03-bf95-40b7aece69bf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:14.992321 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9de3d-5a99-4d03-bf95-40b7aece69bf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:14.992600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9de3d-5a99-4d03-bf95-40b7aece69bf None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:14.993355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a9de3d-5a99-4d03-bf95-40b7aece69bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:14.993668 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a9de3d-5a99-4d03-bf95-40b7aece69bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:14.995170 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a9de3d-5a99-4d03-bf95-40b7aece69bf None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:15.013332 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 250/491] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:14:15.014908 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9de3d-5a99-4d03-bf95-40b7aece69bf None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:15.044755 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:15.053376 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3b1ef-5c13-4370-bac7-9b011d5f8dfe None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:15.098559 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df078c8d-0d4b-4ea0-9ac2-c116a2ba6c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:15.099226 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df078c8d-0d4b-4ea0-9ac2-c116a2ba6c96 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:15.099507 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df078c8d-0d4b-4ea0-9ac2-c116a2ba6c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:15.099776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df078c8d-0d4b-4ea0-9ac2-c116a2ba6c96 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:15.111452 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df078c8d-0d4b-4ea0-9ac2-c116a2ba6c96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:15.116496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-df078c8d-0d4b-4ea0-9ac2-c116a2ba6c96 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:15.116496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:15.116496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:15.116496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:15.116496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:15.116496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:15.116496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:15.116496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:15.116496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:15.116496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:15.116496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:15.116496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:15.116496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:15.116496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:15.116496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:15.116496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:15.117448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:15.117448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:15.117448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:15.117448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:15.117448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:15.117448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:15.117448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:15.117448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:15.117448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:15.117448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:15.117448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:15.117448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:15.117448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:15.117448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:15.117448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:15.117448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:15.118238 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:15.118238 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:15.118238 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:15.118238 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:15.118238 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:15.118238 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:15.118238 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:15.118238 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:15.118238 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:15.118238 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:15.118238 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:15.118238 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:15.118238 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 242/492] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Sep 2 09:14:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:15.119724 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df078c8d-0d4b-4ea0-9ac2-c116a2ba6c96 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:15.134244 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9de3d-5a99-4d03-bf95-40b7aece69bf None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:15.164164 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:15.177507 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685a550-3d4e-4d80-8bdb-036351b08352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:15.184582 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685a550-3d4e-4d80-8bdb-036351b08352 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:15.184582 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685a550-3d4e-4d80-8bdb-036351b08352 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:15.184582 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685a550-3d4e-4d80-8bdb-036351b08352 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:15.184582 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685a550-3d4e-4d80-8bdb-036351b08352 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:15.184582 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685a550-3d4e-4d80-8bdb-036351b08352 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:15.185656 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685a550-3d4e-4d80-8bdb-036351b08352 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:15.201095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 251/493] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Sep 2 09:14:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:15.205492 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c5279-6429-4ba9-acfa-74de61c62867 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:15.205492 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c5279-6429-4ba9-acfa-74de61c62867 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:15.205492 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c5279-6429-4ba9-acfa-74de61c62867 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:15.206830 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 252/494] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:15.208897 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c5279-6429-4ba9-acfa-74de61c62867 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:15.231084 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df078c8d-0d4b-4ea0-9ac2-c116a2ba6c96 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:15.268496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd3a85-9b72-4e7d-864f-5d15c08300e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:15.269048 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd3a85-9b72-4e7d-864f-5d15c08300e3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:15.269329 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd3a85-9b72-4e7d-864f-5d15c08300e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:15.269903 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd3a85-9b72-4e7d-864f-5d15c08300e3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:15.277455 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd3a85-9b72-4e7d-864f-5d15c08300e3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:15.278929 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd3a85-9b72-4e7d-864f-5d15c08300e3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:15.292963 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-20dd3a85-9b72-4e7d-864f-5d15c08300e3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 02 09:14:15.292963 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:15.292963 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:15.292963 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:15.292963 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:15.292963 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:15.292963 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:15.292963 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:15.292963 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:15.292963 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:15.292963 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:15.292963 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:15.292963 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:14:15.292963 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:14:15.292963 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:14:15.292963 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:14:15.293906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:15.293906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:15.293906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:14:15.293906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:15.293906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:14:15.293906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:15.293906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:15.293906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:15.293906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:15.293906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:15.293906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:15.293906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:15.293906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:15.293906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:15.293906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:15.293906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:15.293906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:15.293906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:15.299640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:15.299640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:15.299640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:14:15.299640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:14:15.299640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:14:15.299640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:14:15.299640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:14:15.299640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:14:15.299640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 02 09:14:15.299640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:15.299640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 243/495] 10.222.0.57 () {56 vars in 1093 bytes} [Fri Sep 2 09:14:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:15.299640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd3a85-9b72-4e7d-864f-5d15c08300e3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:15.312297 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:15.330454 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c5279-6429-4ba9-acfa-74de61c62867 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:15.369008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ba9d6-c096-4e0a-8c93-60fc0088ec67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:15.369590 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ba9d6-c096-4e0a-8c93-60fc0088ec67 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:15.369869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ba9d6-c096-4e0a-8c93-60fc0088ec67 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:15.370144 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ba9d6-c096-4e0a-8c93-60fc0088ec67 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:15.370848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ba9d6-c096-4e0a-8c93-60fc0088ec67 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:15.371155 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ba9d6-c096-4e0a-8c93-60fc0088ec67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:15.372508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2ba9d6-c096-4e0a-8c93-60fc0088ec67 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:15.381144 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 253/496] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Sep 2 09:14:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:15.387122 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935962d6-36b1-4dec-999c-f4a92150631d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:15.387412 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935962d6-36b1-4dec-999c-f4a92150631d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:15.387690 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935962d6-36b1-4dec-999c-f4a92150631d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:15.400279 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:15.405082 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd3a85-9b72-4e7d-864f-5d15c08300e3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:15.406223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-935962d6-36b1-4dec-999c-f4a92150631d None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:15.407432 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-935962d6-36b1-4dec-999c-f4a92150631d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:15.439860 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7fca5-fd53-4e30-9247-4b1d60ebcf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:15.440488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7fca5-fd53-4e30-9247-4b1d60ebcf53 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:15.440771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7fca5-fd53-4e30-9247-4b1d60ebcf53 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:15.441044 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7fca5-fd53-4e30-9247-4b1d60ebcf53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:15.453300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac7fca5-fd53-4e30-9247-4b1d60ebcf53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:15.455039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac7fca5-fd53-4e30-9247-4b1d60ebcf53 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:15.469549 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-eac7fca5-fd53-4e30-9247-4b1d60ebcf53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:15.469549 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:15.469549 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:15.469549 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:15.469549 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:15.469549 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:15.469549 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:15.469549 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:15.469549 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:15.469549 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:15.469549 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:15.469549 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:15.469549 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:15.469549 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:15.469549 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:15.469549 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:15.470449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:15.470449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:15.470449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:15.470449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:15.470449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:15.470449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:15.470449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:15.470449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:15.470449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:15.470449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:15.470449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:15.470449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:15.470449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:15.470449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:15.470449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:15.470449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:15.470449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:15.471311 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:15.471311 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:15.471311 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:15.471311 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 244/497] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:15.471533 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7fca5-fd53-4e30-9247-4b1d60ebcf53 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:15.486739 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 254/498] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:15.486739 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935962d6-36b1-4dec-999c-f4a92150631d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:15.584336 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7fca5-fd53-4e30-9247-4b1d60ebcf53 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:15.589483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:15.613627 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935962d6-36b1-4dec-999c-f4a92150631d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:15.621631 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e67468-667d-4428-b9d4-4d6fc5935d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:15.622240 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e67468-667d-4428-b9d4-4d6fc5935d0a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:15.622527 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e67468-667d-4428-b9d4-4d6fc5935d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:15.622835 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e67468-667d-4428-b9d4-4d6fc5935d0a None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:15.623566 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e67468-667d-4428-b9d4-4d6fc5935d0a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:15.623874 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e67468-667d-4428-b9d4-4d6fc5935d0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:15.625343 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e67468-667d-4428-b9d4-4d6fc5935d0a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:15.652362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706783c-3a71-490c-b44c-77c7586d6dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:15.653609 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706783c-3a71-490c-b44c-77c7586d6dfd None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:15.654112 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706783c-3a71-490c-b44c-77c7586d6dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:15.654575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706783c-3a71-490c-b44c-77c7586d6dfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:15.661886 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 245/499] 10.222.0.57 () {56 vars in 1133 bytes} [Fri Sep 2 09:14:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:15.662532 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e67468-667d-4428-b9d4-4d6fc5935d0a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:15.669016 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5706783c-3a71-490c-b44c-77c7586d6dfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:15.670463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5706783c-3a71-490c-b44c-77c7586d6dfd None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:15.676973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:15.686163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-5706783c-3a71-490c-b44c-77c7586d6dfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:15.686163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:15.686163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:15.686163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:15.686163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:15.686163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:15.686163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:15.686163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:15.686163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:15.686163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:15.686163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:15.686163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:15.686163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:15.686163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:15.686163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:15.686163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:15.687094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:15.687094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:15.687094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:15.687094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:15.687094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:15.687094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:15.687094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:15.687094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:15.687094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:15.687094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:15.687094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:15.687094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:15.687094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:15.687094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:15.687094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:15.687094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:15.687094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:15.688065 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:15.688065 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:15.688065 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:15.691858 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 255/500] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:15.693018 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f1287-6f29-4ed9-966e-709a38439dc1 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:15.694534 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f1287-6f29-4ed9-966e-709a38439dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:15.694534 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f1287-6f29-4ed9-966e-709a38439dc1 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:15.698895 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 256/501] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:15.699837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f1287-6f29-4ed9-966e-709a38439dc1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:15.768604 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e67468-667d-4428-b9d4-4d6fc5935d0a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:15.803806 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b1a22-1d1b-4adb-bf7c-3f8e17ac4f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:15.804415 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f1287-6f29-4ed9-966e-709a38439dc1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:15.804674 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b1a22-1d1b-4adb-bf7c-3f8e17ac4f02 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:15.804961 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b1a22-1d1b-4adb-bf7c-3f8e17ac4f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:15.805236 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b1a22-1d1b-4adb-bf7c-3f8e17ac4f02 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:15.805878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b1a22-1d1b-4adb-bf7c-3f8e17ac4f02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:15.806205 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b1a22-1d1b-4adb-bf7c-3f8e17ac4f02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:15.807635 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b1a22-1d1b-4adb-bf7c-3f8e17ac4f02 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:15.818620 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 246/502] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:15.819431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b1a22-1d1b-4adb-bf7c-3f8e17ac4f02 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:15.837340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:15.846635 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72c990-dcbd-440c-b92b-b26a057bd0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:15.847279 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72c990-dcbd-440c-b92b-b26a057bd0a5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:15.847556 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72c990-dcbd-440c-b92b-b26a057bd0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:15.847823 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72c990-dcbd-440c-b92b-b26a057bd0a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:15.887138 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:15.917606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e72c990-dcbd-440c-b92b-b26a057bd0a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:15.919084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e72c990-dcbd-440c-b92b-b26a057bd0a5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:15.921258 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b1a22-1d1b-4adb-bf7c-3f8e17ac4f02 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:15.957349 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377494db-dc9d-4094-bcc6-831ee61f35e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:15.957949 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377494db-dc9d-4094-bcc6-831ee61f35e1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:15.958234 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377494db-dc9d-4094-bcc6-831ee61f35e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:15.958510 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377494db-dc9d-4094-bcc6-831ee61f35e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:15.959185 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377494db-dc9d-4094-bcc6-831ee61f35e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:15.959485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377494db-dc9d-4094-bcc6-831ee61f35e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:15.961445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377494db-dc9d-4094-bcc6-831ee61f35e1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:15.972884 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 247/503] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:15.974113 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb829455-132e-4590-be79-12acdc17234a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:15.974399 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb829455-132e-4590-be79-12acdc17234a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:15.974660 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb829455-132e-4590-be79-12acdc17234a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:15.991485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-fb829455-132e-4590-be79-12acdc17234a None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:15.992673 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fb829455-132e-4590-be79-12acdc17234a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:16.016210 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 257/504] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:14:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:14:16.017119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72c990-dcbd-440c-b92b-b26a057bd0a5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:16.046670 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:16.057229 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 248/505] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:16.058300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb829455-132e-4590-be79-12acdc17234a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:16.120202 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72c990-dcbd-440c-b92b-b26a057bd0a5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:16.123056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:16.159436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1244b-11c5-4fc1-a247-de6a57fa700c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:16.160450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1244b-11c5-4fc1-a247-de6a57fa700c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:16.161031 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1244b-11c5-4fc1-a247-de6a57fa700c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:16.161280 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1244b-11c5-4fc1-a247-de6a57fa700c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:16.165280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb829455-132e-4590-be79-12acdc17234a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:16.174010 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1244b-11c5-4fc1-a247-de6a57fa700c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:16.180702 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-d9f1244b-11c5-4fc1-a247-de6a57fa700c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:16.180702 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:16.180702 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:16.180702 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:16.180702 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:16.180702 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:16.180702 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:16.180702 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:16.180702 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:16.180702 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:16.180702 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:16.180702 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:16.180702 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:16.180702 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:16.180702 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:16.180702 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:16.181565 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:16.181565 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:16.181565 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:16.181565 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:16.181565 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:16.181565 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:16.181565 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:16.181565 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:16.181565 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:16.181565 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:16.181565 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:16.181565 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:16.181565 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:16.181565 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:16.181565 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:16.181565 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:16.187480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:16.187480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:16.187480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:16.187480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:16.187480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:16.187480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:16.187480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:16.187480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:16.187480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:16.187480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:16.187480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:16.187480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:16.187480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 258/506] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Sep 2 09:14:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:16.187480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1244b-11c5-4fc1-a247-de6a57fa700c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:16.198725 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19553e2d-3617-4c7e-8823-55a089b0e29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:16.199617 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19553e2d-3617-4c7e-8823-55a089b0e29c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:16.199894 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19553e2d-3617-4c7e-8823-55a089b0e29c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:16.200164 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19553e2d-3617-4c7e-8823-55a089b0e29c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:16.207125 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19553e2d-3617-4c7e-8823-55a089b0e29c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:16.208551 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19553e2d-3617-4c7e-8823-55a089b0e29c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:16.222149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-19553e2d-3617-4c7e-8823-55a089b0e29c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 02 09:14:16.222149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:16.222149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:16.222149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:16.222149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:16.222149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:16.222149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:16.222149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:16.222149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:16.222149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:16.222149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:16.222149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:16.222149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:14:16.222149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:14:16.222149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:14:16.222149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:14:16.223301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:16.223301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:16.223301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:14:16.223301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:16.223301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:14:16.223301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:16.223301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:16.223301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:16.223301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:16.223301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:16.223301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:16.223301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:16.223301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:16.223301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:16.223301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:16.223301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:16.223301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:16.223301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:16.229274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:16.229274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:16.229274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:14:16.229274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:14:16.229274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:14:16.229274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:14:16.229274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:14:16.229274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:14:16.229274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 02 09:14:16.229274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:16.229274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 249/507] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:14:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:16.229274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19553e2d-3617-4c7e-8823-55a089b0e29c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:16.289620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:16.300158 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1244b-11c5-4fc1-a247-de6a57fa700c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:16.331565 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19553e2d-3617-4c7e-8823-55a089b0e29c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:16.335558 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286caac4-46a8-425b-a20a-44d64d083985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:16.336705 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286caac4-46a8-425b-a20a-44d64d083985 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:16.336981 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286caac4-46a8-425b-a20a-44d64d083985 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:16.337247 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286caac4-46a8-425b-a20a-44d64d083985 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:16.344142 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286caac4-46a8-425b-a20a-44d64d083985 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:16.345569 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286caac4-46a8-425b-a20a-44d64d083985 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:16.348604 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:16.358542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-286caac4-46a8-425b-a20a-44d64d083985 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 02 09:14:16.358542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:16.358542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:16.358542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:16.358542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:16.358542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:16.358542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:16.358542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:16.358542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:16.358542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:16.358542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:16.358542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:16.358542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:14:16.358542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:14:16.358542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:14:16.358542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:14:16.359747 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:16.359747 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:16.359747 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:14:16.359747 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:16.359747 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:14:16.359747 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:16.359747 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:16.359747 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:16.359747 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:16.359747 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:16.359747 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:16.359747 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:16.359747 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:16.359747 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:16.359747 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:16.359747 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:16.359747 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:16.359747 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:16.360663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:16.360663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:16.360663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:14:16.360663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:14:16.360663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:14:16.360663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:14:16.360663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:14:16.360663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:14:16.360663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 02 09:14:16.360663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:16.360663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 259/508] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:14:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:16.360663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286caac4-46a8-425b-a20a-44d64d083985 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:16.369798 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df98ec2-7701-41e1-82dc-10b6885f9a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:16.370476 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df98ec2-7701-41e1-82dc-10b6885f9a2a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:16.370729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df98ec2-7701-41e1-82dc-10b6885f9a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:16.371015 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df98ec2-7701-41e1-82dc-10b6885f9a2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:16.381523 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df98ec2-7701-41e1-82dc-10b6885f9a2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:16.383034 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df98ec2-7701-41e1-82dc-10b6885f9a2a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:16.396997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-9df98ec2-7701-41e1-82dc-10b6885f9a2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:16.396997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:16.396997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:16.396997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:16.396997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:16.396997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:16.396997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:16.396997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:16.396997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:16.396997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:16.396997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:16.396997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:16.396997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:16.396997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:16.396997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:16.396997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:16.397978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:16.397978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:16.397978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:16.397978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:16.397978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:16.397978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:16.397978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:16.397978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:16.397978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:16.397978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:16.397978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:16.397978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:16.397978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:16.397978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:16.397978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:16.397978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:16.397978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:16.399355 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:16.399355 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:16.399355 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:16.399355 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 250/509] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:16.399355 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df98ec2-7701-41e1-82dc-10b6885f9a2a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:16.465927 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286caac4-46a8-425b-a20a-44d64d083985 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:16.496607 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df98ec2-7701-41e1-82dc-10b6885f9a2a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:16.497790 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9473fa9-7dec-4d02-ad67-5f1d845ce9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:16.498384 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9473fa9-7dec-4d02-ad67-5f1d845ce9dc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:16.498666 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9473fa9-7dec-4d02-ad67-5f1d845ce9dc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:16.498960 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9473fa9-7dec-4d02-ad67-5f1d845ce9dc None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:16.499612 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9473fa9-7dec-4d02-ad67-5f1d845ce9dc None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:16.499913 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9473fa9-7dec-4d02-ad67-5f1d845ce9dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:16.501272 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9473fa9-7dec-4d02-ad67-5f1d845ce9dc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:16.509020 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 260/510] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Sep 2 09:14:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:16.509809 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9473fa9-7dec-4d02-ad67-5f1d845ce9dc None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:16.524257 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:16.530150 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d40dcfb-7dde-4bb3-985b-165ad399cfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:16.530783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40dcfb-7dde-4bb3-985b-165ad399cfc8 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:16.531066 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40dcfb-7dde-4bb3-985b-165ad399cfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:16.531333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40dcfb-7dde-4bb3-985b-165ad399cfc8 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:16.532057 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d40dcfb-7dde-4bb3-985b-165ad399cfc8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:16.532358 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d40dcfb-7dde-4bb3-985b-165ad399cfc8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:16.533848 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d40dcfb-7dde-4bb3-985b-165ad399cfc8 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:16.551382 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 251/511] 10.222.0.57 () {56 vars in 1137 bytes} [Fri Sep 2 09:14:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:16.551984 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d40dcfb-7dde-4bb3-985b-165ad399cfc8 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:16.578600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:16.612665 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9473fa9-7dec-4d02-ad67-5f1d845ce9dc None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:16.645818 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe48c06-8748-4161-87ab-5ec01433b565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:16.646377 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe48c06-8748-4161-87ab-5ec01433b565 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:16.646660 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe48c06-8748-4161-87ab-5ec01433b565 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:16.646965 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe48c06-8748-4161-87ab-5ec01433b565 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:16.650257 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d40dcfb-7dde-4bb3-985b-165ad399cfc8 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:16.657850 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe48c06-8748-4161-87ab-5ec01433b565 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:16.659329 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe48c06-8748-4161-87ab-5ec01433b565 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:16.673064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2fe48c06-8748-4161-87ab-5ec01433b565 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:16.673064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:16.673064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:16.673064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:16.673064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:16.673064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:16.673064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:16.673064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:16.673064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:16.673064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:16.673064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:16.673064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:16.673064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:16.673064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:16.673064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:16.673064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:16.673943 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:16.673943 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:16.673943 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:16.673943 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:16.673943 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:16.673943 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:16.673943 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:16.673943 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:16.673943 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:16.673943 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:16.673943 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:16.673943 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:16.673943 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:16.673943 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:16.673943 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:16.673943 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:16.673943 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:16.674825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:16.674825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:16.674825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:16.674825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 261/512] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:16.675063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe48c06-8748-4161-87ab-5ec01433b565 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:16.683333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102de574-8e9b-4a3d-9b9d-2a4454f21f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:16.683971 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102de574-8e9b-4a3d-9b9d-2a4454f21f5d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:16.684251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102de574-8e9b-4a3d-9b9d-2a4454f21f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:16.684525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102de574-8e9b-4a3d-9b9d-2a4454f21f5d None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:16.685245 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102de574-8e9b-4a3d-9b9d-2a4454f21f5d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:16.685555 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102de574-8e9b-4a3d-9b9d-2a4454f21f5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:16.686997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102de574-8e9b-4a3d-9b9d-2a4454f21f5d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:16.704230 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 252/513] 10.222.0.57 () {56 vars in 1137 bytes} [Fri Sep 2 09:14:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:16.705017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102de574-8e9b-4a3d-9b9d-2a4454f21f5d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:16.736287 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:16.772819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe48c06-8748-4161-87ab-5ec01433b565 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:16.787095 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:16.801315 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102de574-8e9b-4a3d-9b9d-2a4454f21f5d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:16.806496 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e248d-b315-4091-806e-def6650b0669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:16.807131 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e248d-b315-4091-806e-def6650b0669 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:16.807414 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e248d-b315-4091-806e-def6650b0669 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:16.807687 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e248d-b315-4091-806e-def6650b0669 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:16.808330 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e248d-b315-4091-806e-def6650b0669 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:16.808637 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e248d-b315-4091-806e-def6650b0669 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:16.809996 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70e248d-b315-4091-806e-def6650b0669 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:16.822101 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 262/514] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:16.823093 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e248d-b315-4091-806e-def6650b0669 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:16.837098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d37c83-956b-4245-8832-275d0b71c49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:16.837720 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d37c83-956b-4245-8832-275d0b71c49f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:16.838001 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d37c83-956b-4245-8832-275d0b71c49f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:16.838271 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d37c83-956b-4245-8832-275d0b71c49f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:16.848409 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d37c83-956b-4245-8832-275d0b71c49f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:16.849848 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d37c83-956b-4245-8832-275d0b71c49f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:16.864547 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-c6d37c83-956b-4245-8832-275d0b71c49f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:16.864547 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:16.864547 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:16.864547 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:16.864547 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:16.864547 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:16.864547 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:16.864547 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:16.864547 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:16.864547 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:16.864547 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:16.864547 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:16.864547 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:16.864547 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:16.864547 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:16.864547 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:16.865500 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:16.865500 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:16.865500 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:16.865500 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:16.865500 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:16.865500 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:16.865500 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:16.865500 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:16.865500 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:16.865500 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:16.865500 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:16.865500 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:16.865500 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:16.865500 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:16.865500 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:16.865500 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:16.865500 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:16.866362 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:16.866362 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:16.866362 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:16.866362 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 253/515] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:16.866362 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d37c83-956b-4245-8832-275d0b71c49f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:16.926929 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70e248d-b315-4091-806e-def6650b0669 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:16.950988 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:16.962166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b39d2e-c024-4f11-89be-67d93d6b653f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:16.962797 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b39d2e-c024-4f11-89be-67d93d6b653f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:16.963077 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b39d2e-c024-4f11-89be-67d93d6b653f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:16.964584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b39d2e-c024-4f11-89be-67d93d6b653f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:16.964679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d37c83-956b-4245-8832-275d0b71c49f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:16.997546 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8b896-0b08-4308-bc8a-86ed5a308112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:16.998143 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8b896-0b08-4308-bc8a-86ed5a308112 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:16.998426 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8b896-0b08-4308-bc8a-86ed5a308112 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:16.998699 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8b896-0b08-4308-bc8a-86ed5a308112 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:16.999359 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb8b896-0b08-4308-bc8a-86ed5a308112 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:16.999660 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb8b896-0b08-4308-bc8a-86ed5a308112 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:17.001031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb8b896-0b08-4308-bc8a-86ed5a308112 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:17.007705 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:17.012638 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 254/516] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:17.013414 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8b896-0b08-4308-bc8a-86ed5a308112 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:17.030235 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b39d2e-c024-4f11-89be-67d93d6b653f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:17.031719 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b39d2e-c024-4f11-89be-67d93d6b653f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:17.108578 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8b896-0b08-4308-bc8a-86ed5a308112 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:17.129579 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 263/517] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:14:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:14:17.130401 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b39d2e-c024-4f11-89be-67d93d6b653f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:17.142735 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80218f-ed2a-4dbd-bc6c-dc181afcc025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:17.143331 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80218f-ed2a-4dbd-bc6c-dc181afcc025 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:17.143607 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80218f-ed2a-4dbd-bc6c-dc181afcc025 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:17.143876 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80218f-ed2a-4dbd-bc6c-dc181afcc025 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:17.154368 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:17.207464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:17.211285 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80218f-ed2a-4dbd-bc6c-dc181afcc025 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:17.212755 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80218f-ed2a-4dbd-bc6c-dc181afcc025 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:17.228256 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b39d2e-c024-4f11-89be-67d93d6b653f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:17.261384 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6d9b8-a27d-4ae8-a911-67d94b3ac3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:17.262006 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6d9b8-a27d-4ae8-a911-67d94b3ac3d9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:17.262285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6d9b8-a27d-4ae8-a911-67d94b3ac3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:17.262556 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6d9b8-a27d-4ae8-a911-67d94b3ac3d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:17.268936 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6d9b8-a27d-4ae8-a911-67d94b3ac3d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:17.270369 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6d9b8-a27d-4ae8-a911-67d94b3ac3d9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:17.282283 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-d0c6d9b8-a27d-4ae8-a911-67d94b3ac3d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:17.282283 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:17.282283 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:17.282283 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:17.282283 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:17.282283 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:17.282283 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:17.282283 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:17.282283 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:17.282283 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:17.282283 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:17.282283 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:17.282283 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:14:17.282283 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:14:17.282283 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:14:17.283434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:14:17.283434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:17.283434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:17.283434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:17.283434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:17.283434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:17.283434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:17.283434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:17.283434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:17.283434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:17.283434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:17.283434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:17.283434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:17.283434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:17.283434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:17.283434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:17.283434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:17.284269 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:14:17.284269 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:14:17.284269 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:14:17.284269 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:14:17.284269 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:14:17.284269 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:14:17.284269 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:17.284269 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:17.284269 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 264/518] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:17.284269 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6d9b8-a27d-4ae8-a911-67d94b3ac3d9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:17.311077 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 255/519] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:14:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:14:17.311887 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80218f-ed2a-4dbd-bc6c-dc181afcc025 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:17.373332 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:17.388900 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6d9b8-a27d-4ae8-a911-67d94b3ac3d9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:17.405146 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:17.408506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80218f-ed2a-4dbd-bc6c-dc181afcc025 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:17.422262 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a3584-f712-48b4-a2cf-ef4b3f5d2fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:17.422895 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a3584-f712-48b4-a2cf-ef4b3f5d2fd9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:17.423181 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a3584-f712-48b4-a2cf-ef4b3f5d2fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:17.423455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a3584-f712-48b4-a2cf-ef4b3f5d2fd9 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:17.424096 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a3584-f712-48b4-a2cf-ef4b3f5d2fd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:17.424403 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a3584-f712-48b4-a2cf-ef4b3f5d2fd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:17.425768 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a3584-f712-48b4-a2cf-ef4b3f5d2fd9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:17.437421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 265/520] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:17.438204 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a3584-f712-48b4-a2cf-ef4b3f5d2fd9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:17.440458 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095180d-e768-4f16-9dcb-3a83120678e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:17.441096 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095180d-e768-4f16-9dcb-3a83120678e5 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:17.441377 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095180d-e768-4f16-9dcb-3a83120678e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:17.441700 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4095180d-e768-4f16-9dcb-3a83120678e5 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:17.442298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4095180d-e768-4f16-9dcb-3a83120678e5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:17.443743 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4095180d-e768-4f16-9dcb-3a83120678e5 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:17.532891 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 256/521] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Sep 2 09:14:17 2022] POST /identity/v3/users => generated 247 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:14:17.533646 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095180d-e768-4f16-9dcb-3a83120678e5 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:17.534302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a3584-f712-48b4-a2cf-ef4b3f5d2fd9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:17.577810 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3680c05-6db2-41e8-a862-14e210e2e935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:17.578475 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3680c05-6db2-41e8-a862-14e210e2e935 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:17.578787 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3680c05-6db2-41e8-a862-14e210e2e935 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:17.579068 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3680c05-6db2-41e8-a862-14e210e2e935 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:17.593050 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3680c05-6db2-41e8-a862-14e210e2e935 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:17.595784 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3680c05-6db2-41e8-a862-14e210e2e935 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:17.597706 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:17.611124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-d3680c05-6db2-41e8-a862-14e210e2e935 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:17.611124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:17.611124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:17.611124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:17.611124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:17.611124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:17.611124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:17.611124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:17.611124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:17.611124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:17.611124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:17.611124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:17.611124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:17.611124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:17.611124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:17.611124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:17.617100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:17.617100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:17.617100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:17.617100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:17.617100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:17.617100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:17.617100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:17.617100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:17.617100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:17.617100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:17.617100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:17.617100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:17.617100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:17.617100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:17.617100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:17.617100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:17.617100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:17.619625 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:17.619625 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:17.619625 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:17.619625 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 266/522] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:17.619625 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3680c05-6db2-41e8-a862-14e210e2e935 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:17.660942 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4095180d-e768-4f16-9dcb-3a83120678e5 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:17.701857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d421d-e502-4303-a5d5-c6c63fce3f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:17.702474 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d421d-e502-4303-a5d5-c6c63fce3f19 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:17.702790 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d421d-e502-4303-a5d5-c6c63fce3f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:17.703069 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95d421d-e502-4303-a5d5-c6c63fce3f19 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:17.703769 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d421d-e502-4303-a5d5-c6c63fce3f19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:17.704084 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d421d-e502-4303-a5d5-c6c63fce3f19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:17.705513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95d421d-e502-4303-a5d5-c6c63fce3f19 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:17.724335 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 257/523] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:14:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:17.725027 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d421d-e502-4303-a5d5-c6c63fce3f19 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:17.736230 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3680c05-6db2-41e8-a862-14e210e2e935 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:17.740920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:17.776408 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5ca388-4f33-4b78-966f-839497f59c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:17.777042 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ca388-4f33-4b78-966f-839497f59c0a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:17.777332 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ca388-4f33-4b78-966f-839497f59c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:17.777607 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ca388-4f33-4b78-966f-839497f59c0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:17.789534 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5ca388-4f33-4b78-966f-839497f59c0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:17.790939 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5ca388-4f33-4b78-966f-839497f59c0a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:17.810001 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-0f5ca388-4f33-4b78-966f-839497f59c0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:17.810001 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:17.810001 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:17.810001 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:17.810001 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:17.810001 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:17.810001 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:17.810001 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:17.810001 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:17.810001 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:17.810001 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:17.810001 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:17.810001 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:17.810001 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:17.810001 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:17.810001 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:17.811217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:17.811217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:17.811217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:17.811217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:17.811217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:17.811217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:17.811217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:17.811217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:17.811217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:17.811217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:17.811217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:17.811217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:17.811217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:17.811217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:17.811217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:17.811217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:17.811217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:17.812069 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:17.812069 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:17.812069 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:17.814844 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 267/524] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:17.815885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5ca388-4f33-4b78-966f-839497f59c0a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:17.831326 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95d421d-e502-4303-a5d5-c6c63fce3f19 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:17.875316 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:17.876363 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0f269-6503-46dd-8764-fc1a84ba288d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:17.877319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0f269-6503-46dd-8764-fc1a84ba288d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:17.877610 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0f269-6503-46dd-8764-fc1a84ba288d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:17.877884 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0f269-6503-46dd-8764-fc1a84ba288d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:17.886808 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f0f269-6503-46dd-8764-fc1a84ba288d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:17.888310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f0f269-6503-46dd-8764-fc1a84ba288d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:17.901366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-10f0f269-6503-46dd-8764-fc1a84ba288d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 02 09:14:17.901366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:17.901366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:17.901366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:17.901366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:17.901366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:17.901366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:17.901366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:17.901366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:17.901366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:17.901366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:17.901366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:17.901366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:14:17.901366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:14:17.901366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:14:17.901366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:14:17.902298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:17.902298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:17.902298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:14:17.902298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:17.902298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:14:17.902298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:17.902298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:17.902298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:17.902298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:17.902298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:17.902298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:17.902298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:17.902298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:17.902298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:17.902298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:17.902298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:17.902298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:17.902298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:17.903319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:17.903319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:17.903319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:14:17.903319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:14:17.903319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:14:17.903319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:14:17.903319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:14:17.903319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:14:17.903319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 02 09:14:17.903319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:17.907221 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 258/525] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Sep 2 09:14:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:17.908152 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0f269-6503-46dd-8764-fc1a84ba288d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:17.932471 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5ca388-4f33-4b78-966f-839497f59c0a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:17.970269 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ecd9e0-98f6-4616-a11c-6c6167c78d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:17.970935 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecd9e0-98f6-4616-a11c-6c6167c78d00 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:17.971221 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecd9e0-98f6-4616-a11c-6c6167c78d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:17.971496 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecd9e0-98f6-4616-a11c-6c6167c78d00 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:17.972144 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ecd9e0-98f6-4616-a11c-6c6167c78d00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:17.972448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ecd9e0-98f6-4616-a11c-6c6167c78d00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:17.973843 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ecd9e0-98f6-4616-a11c-6c6167c78d00 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:17.991448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 268/526] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:17.991448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ecd9e0-98f6-4616-a11c-6c6167c78d00 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:17.992917 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:18.019689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0f269-6503-46dd-8764-fc1a84ba288d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:18.056566 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cac561-74a8-4904-bb65-672bd7d4fb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:18.057092 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cac561-74a8-4904-bb65-672bd7d4fb3b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:18.057374 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cac561-74a8-4904-bb65-672bd7d4fb3b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:18.057650 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cac561-74a8-4904-bb65-672bd7d4fb3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:18.058374 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cac561-74a8-4904-bb65-672bd7d4fb3b None admin] RBAC: Adding query filter params (role.id=faf1138f11d34abe8a862f1a1f7c140a, scope.project.id=d18a5117a03b4f21ab3290a15680de66, user.id=1da355c8be2d496bb08feb0fe74e04b9) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:18.058692 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cac561-74a8-4904-bb65-672bd7d4fb3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:18.060346 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cac561-74a8-4904-bb65-672bd7d4fb3b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:18.078735 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 259/527] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:14:17 2022] GET /identity/v3/role_assignments?user.id=1da355c8be2d496bb08feb0fe74e04b9&scope.project.id=d18a5117a03b4f21ab3290a15680de66&role.id=faf1138f11d34abe8a862f1a1f7c140a => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:18.078735 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cac561-74a8-4904-bb65-672bd7d4fb3b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:18.100310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ecd9e0-98f6-4616-a11c-6c6167c78d00 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:18.117199 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:18.156070 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a3fbd-96a5-4a3e-8bb6-196b3957348a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:18.157653 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a3fbd-96a5-4a3e-8bb6-196b3957348a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:18.158196 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a3fbd-96a5-4a3e-8bb6-196b3957348a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:18.158619 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a3fbd-96a5-4a3e-8bb6-196b3957348a None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:18.159759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a3fbd-96a5-4a3e-8bb6-196b3957348a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:18.160750 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a3fbd-96a5-4a3e-8bb6-196b3957348a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:18.162621 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a3fbd-96a5-4a3e-8bb6-196b3957348a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:18.184143 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 269/528] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:18.185881 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a3fbd-96a5-4a3e-8bb6-196b3957348a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:18.211807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cac561-74a8-4904-bb65-672bd7d4fb3b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:18.252348 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71473627-ff9c-4196-963a-cc9d7a9e4030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:18.252965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71473627-ff9c-4196-963a-cc9d7a9e4030 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:18.253245 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71473627-ff9c-4196-963a-cc9d7a9e4030 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:18.253517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71473627-ff9c-4196-963a-cc9d7a9e4030 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:18.254254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71473627-ff9c-4196-963a-cc9d7a9e4030 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:18.254729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71473627-ff9c-4196-963a-cc9d7a9e4030 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:18.256919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71473627-ff9c-4196-963a-cc9d7a9e4030 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:18.267644 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:18.278692 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 260/529] 10.222.0.57 () {56 vars in 1139 bytes} [Fri Sep 2 09:14:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:18.283009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71473627-ff9c-4196-963a-cc9d7a9e4030 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:18.332687 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a3fbd-96a5-4a3e-8bb6-196b3957348a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:18.371955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b092f0-bf40-4168-84e1-56a68dfe804b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:18.372931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b092f0-bf40-4168-84e1-56a68dfe804b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:18.373517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b092f0-bf40-4168-84e1-56a68dfe804b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:18.373993 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b092f0-bf40-4168-84e1-56a68dfe804b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:18.381485 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b092f0-bf40-4168-84e1-56a68dfe804b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:18.383835 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b092f0-bf40-4168-84e1-56a68dfe804b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:18.389750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71473627-ff9c-4196-963a-cc9d7a9e4030 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:18.396157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-47b092f0-bf40-4168-84e1-56a68dfe804b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:18.396157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:18.396157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:18.396157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:18.396157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:18.396157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:18.396157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:18.396157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:18.396157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:18.396157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:18.396157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:18.396157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:18.396157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:14:18.396157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:14:18.396157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:14:18.397034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:14:18.397034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:18.397034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:18.397034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:18.397034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:18.397034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:18.397034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:18.397034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:18.397034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:18.397034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:18.397034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:18.397034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:18.397034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:18.397034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:18.397034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:18.397034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:18.397034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:18.397892 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:14:18.397892 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:14:18.397892 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:14:18.397892 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:14:18.397892 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:14:18.397892 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:14:18.397892 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:18.397892 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:18.400976 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 270/530] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:18.402138 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b092f0-bf40-4168-84e1-56a68dfe804b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:18.425715 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7faee-0cbe-4d64-a987-f79db3d9b6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:18.426351 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7faee-0cbe-4d64-a987-f79db3d9b6c7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:18.426633 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7faee-0cbe-4d64-a987-f79db3d9b6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:18.427001 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7faee-0cbe-4d64-a987-f79db3d9b6c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:18.434279 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7faee-0cbe-4d64-a987-f79db3d9b6c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:18.435755 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7faee-0cbe-4d64-a987-f79db3d9b6c7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:18.443910 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:18.450148 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-8ad7faee-0cbe-4d64-a987-f79db3d9b6c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:18.450148 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:18.450148 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:18.450148 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:18.450148 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:18.450148 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:18.450148 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:18.450148 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:18.450148 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:18.450148 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:18.450148 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:18.450148 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:18.450148 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:14:18.450148 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:14:18.450148 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:14:18.451081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:14:18.451081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:18.451081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:18.451081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:18.451081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:18.451081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:18.451081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:18.451081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:18.451081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:18.451081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:18.451081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:18.451081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:18.451081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:18.451081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:18.451081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:18.451081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:18.451081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:18.451925 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:14:18.451925 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:14:18.451925 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:14:18.451925 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:14:18.451925 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:14:18.451925 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:14:18.451925 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:18.451925 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:18.458588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 261/531] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:18.459672 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7faee-0cbe-4d64-a987-f79db3d9b6c7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:18.523900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:18.535030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b092f0-bf40-4168-84e1-56a68dfe804b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:18.578583 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ead0fb-bfdb-4b41-bb43-5bc87b732248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:18.578807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7faee-0cbe-4d64-a987-f79db3d9b6c7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:18.579276 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ead0fb-bfdb-4b41-bb43-5bc87b732248 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:18.579562 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ead0fb-bfdb-4b41-bb43-5bc87b732248 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:18.579832 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ead0fb-bfdb-4b41-bb43-5bc87b732248 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:18.622769 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0362db9-9221-43db-b64e-0296b4c8e7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:18.624408 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0362db9-9221-43db-b64e-0296b4c8e7ce None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:18.624757 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0362db9-9221-43db-b64e-0296b4c8e7ce None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:18.625042 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0362db9-9221-43db-b64e-0296b4c8e7ce None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:18.625809 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0362db9-9221-43db-b64e-0296b4c8e7ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:18.626122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0362db9-9221-43db-b64e-0296b4c8e7ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:18.627645 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0362db9-9221-43db-b64e-0296b4c8e7ce None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:18.644222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 262/532] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:14:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:18.645305 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0362db9-9221-43db-b64e-0296b4c8e7ce None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:18.669023 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ead0fb-bfdb-4b41-bb43-5bc87b732248 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:18.671046 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ead0fb-bfdb-4b41-bb43-5bc87b732248 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:18.705894 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:18.767166 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0362db9-9221-43db-b64e-0296b4c8e7ce None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:18.783683 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 271/533] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:14:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:14:18.784534 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ead0fb-bfdb-4b41-bb43-5bc87b732248 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:18.791133 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:18.803642 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e544f6-79fd-474c-8cf0-bfca1c51e66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:18.804342 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e544f6-79fd-474c-8cf0-bfca1c51e66b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:18.804698 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e544f6-79fd-474c-8cf0-bfca1c51e66b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:18.805026 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e544f6-79fd-474c-8cf0-bfca1c51e66b None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:18.805766 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e544f6-79fd-474c-8cf0-bfca1c51e66b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:18.808313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e544f6-79fd-474c-8cf0-bfca1c51e66b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:18.808313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e544f6-79fd-474c-8cf0-bfca1c51e66b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:18.819692 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 263/534] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:14:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:18.825321 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e544f6-79fd-474c-8cf0-bfca1c51e66b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:18.890411 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ead0fb-bfdb-4b41-bb43-5bc87b732248 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:18.918988 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:18.927262 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8f925-f18d-4890-b68c-441e457db527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:18.927884 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8f925-f18d-4890-b68c-441e457db527 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:18.928165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8f925-f18d-4890-b68c-441e457db527 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:18.932836 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8f925-f18d-4890-b68c-441e457db527 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:18.932836 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b8f925-f18d-4890-b68c-441e457db527 None admin] RBAC: Adding query filter params (role.id=faf1138f11d34abe8a862f1a1f7c140a, scope.project.id=d18a5117a03b4f21ab3290a15680de66, user.id=b6a1f44f6fe94c09b696867871fd5b47) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:18.932836 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b8f925-f18d-4890-b68c-441e457db527 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:18.932836 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b8f925-f18d-4890-b68c-441e457db527 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:18.933092 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e544f6-79fd-474c-8cf0-bfca1c51e66b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:18.937033 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 272/535] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:14:18 2022] GET /identity/v3/role_assignments?user.id=b6a1f44f6fe94c09b696867871fd5b47&scope.project.id=d18a5117a03b4f21ab3290a15680de66&role.id=faf1138f11d34abe8a862f1a1f7c140a => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:18.937914 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8f925-f18d-4890-b68c-441e457db527 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:19.011506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d7512-e953-4b9c-9f58-b50e9f7a3525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:19.012156 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d7512-e953-4b9c-9f58-b50e9f7a3525 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:19.012433 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d7512-e953-4b9c-9f58-b50e9f7a3525 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:19.012701 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d7512-e953-4b9c-9f58-b50e9f7a3525 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:19.033480 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d7512-e953-4b9c-9f58-b50e9f7a3525 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:19.034977 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d7512-e953-4b9c-9f58-b50e9f7a3525 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:19.057959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-0c9d7512-e953-4b9c-9f58-b50e9f7a3525 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:19.057959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:19.057959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:19.057959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:19.057959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:19.057959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:19.057959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:19.057959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:19.057959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:19.057959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:19.057959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:19.057959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:19.057959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:19.057959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:19.057959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:19.057959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:19.059024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:19.059024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:19.059024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:19.059024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:19.059024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:19.059024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:19.059024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:19.059024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:19.059024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:19.059024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:19.059024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:19.059024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:19.059024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:19.059024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:19.059024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:19.059024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:19.059024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:19.060852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:19.060852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:19.060852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:19.061242 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8f925-f18d-4890-b68c-441e457db527 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:19.061387 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 264/536] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:19.067902 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d7512-e953-4b9c-9f58-b50e9f7a3525 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:19.096501 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07baf200-1f15-4d6c-acb8-b22ad83157d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:19.097116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07baf200-1f15-4d6c-acb8-b22ad83157d6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:19.097400 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07baf200-1f15-4d6c-acb8-b22ad83157d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:19.097678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07baf200-1f15-4d6c-acb8-b22ad83157d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:19.098468 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07baf200-1f15-4d6c-acb8-b22ad83157d6 None admin] RBAC: Adding query filter params (role.id=faf1138f11d34abe8a862f1a1f7c140a, scope.project.id=d18a5117a03b4f21ab3290a15680de66, user.id=31b555bb94ce44ce918cbf2a012576a7) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:19.098897 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07baf200-1f15-4d6c-acb8-b22ad83157d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:19.100371 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07baf200-1f15-4d6c-acb8-b22ad83157d6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:19.108020 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 273/537] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:14:18 2022] GET /identity/v3/role_assignments?user.id=31b555bb94ce44ce918cbf2a012576a7&scope.project.id=d18a5117a03b4f21ab3290a15680de66&role.id=faf1138f11d34abe8a862f1a1f7c140a => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:19.127325 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:19.222671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d7512-e953-4b9c-9f58-b50e9f7a3525 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:19.259648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1bf24-a98e-499c-bd30-38238533b143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:19.260261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1bf24-a98e-499c-bd30-38238533b143 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:19.260537 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1bf24-a98e-499c-bd30-38238533b143 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:19.260806 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1bf24-a98e-499c-bd30-38238533b143 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:19.261452 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1bf24-a98e-499c-bd30-38238533b143 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:19.261750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1bf24-a98e-499c-bd30-38238533b143 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:19.263160 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1bf24-a98e-499c-bd30-38238533b143 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:19.276868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 265/538] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:19.282089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07baf200-1f15-4d6c-acb8-b22ad83157d6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:19.384834 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:19.451271 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07baf200-1f15-4d6c-acb8-b22ad83157d6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:19.499222 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc271e-9f93-45f9-8ee2-a8d438b7f11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:19.499789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc271e-9f93-45f9-8ee2-a8d438b7f11d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:19.500072 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc271e-9f93-45f9-8ee2-a8d438b7f11d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:19.500349 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc271e-9f93-45f9-8ee2-a8d438b7f11d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:19.511943 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc271e-9f93-45f9-8ee2-a8d438b7f11d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:19.513404 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc271e-9f93-45f9-8ee2-a8d438b7f11d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:19.528548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-28fc271e-9f93-45f9-8ee2-a8d438b7f11d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:19.528548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:19.528548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:19.528548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:19.528548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:19.528548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:19.528548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:19.528548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:19.528548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:19.528548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:19.528548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:19.528548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:19.528548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:14:19.528548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:14:19.528548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:14:19.530679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:14:19.530679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:19.530679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:19.530679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:19.530679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:19.530679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:19.530679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:19.530679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:19.530679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:19.530679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:19.530679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:19.530679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:19.530679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:19.530679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:19.530679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:19.530679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:19.530679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:19.531679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:14:19.531679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:14:19.531679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:14:19.531679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:14:19.531679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:14:19.531679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:14:19.531679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:19.531679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:19.531679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 274/539] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:19.535396 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1bf24-a98e-499c-bd30-38238533b143 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:19.564877 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8078604-6c3f-4f06-ae8d-2392776344ae None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:19.565485 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8078604-6c3f-4f06-ae8d-2392776344ae None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:19.565980 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8078604-6c3f-4f06-ae8d-2392776344ae None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:19.568226 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 275/540] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:19.575738 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65eb931-6b30-4621-9c84-1b8df8905f79 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:19.576307 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65eb931-6b30-4621-9c84-1b8df8905f79 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:19.576778 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65eb931-6b30-4621-9c84-1b8df8905f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:19.613618 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e65eb931-6b30-4621-9c84-1b8df8905f79 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:19.615243 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e65eb931-6b30-4621-9c84-1b8df8905f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:19.709146 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:19.728989 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1bf24-a98e-499c-bd30-38238533b143 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:19.753312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 276/541] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 09:14:19.780528 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329bd881-7a0f-49a1-b58f-47a80ba6a28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:19.781151 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329bd881-7a0f-49a1-b58f-47a80ba6a28f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:19.781438 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329bd881-7a0f-49a1-b58f-47a80ba6a28f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:19.781713 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329bd881-7a0f-49a1-b58f-47a80ba6a28f None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:19.782409 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329bd881-7a0f-49a1-b58f-47a80ba6a28f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:19.782803 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329bd881-7a0f-49a1-b58f-47a80ba6a28f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:19.784302 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329bd881-7a0f-49a1-b58f-47a80ba6a28f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:19.803952 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 266/542] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:14:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:14:19.809497 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65eb931-6b30-4621-9c84-1b8df8905f79 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:19.893709 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:19.962012 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65eb931-6b30-4621-9c84-1b8df8905f79 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:20.009048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f39fa3-5bb0-42db-85a6-944899468ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:20.009594 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f39fa3-5bb0-42db-85a6-944899468ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:20.009884 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f39fa3-5bb0-42db-85a6-944899468ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:20.010158 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f39fa3-5bb0-42db-85a6-944899468ae0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:20.010918 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f39fa3-5bb0-42db-85a6-944899468ae0 None admin] RBAC: Adding query filter params (role.id=faf1138f11d34abe8a862f1a1f7c140a, scope.project.id=d18a5117a03b4f21ab3290a15680de66, user.id=dfea2ee0c9e84b48ac9bafe3173d937a) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:20.011232 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f39fa3-5bb0-42db-85a6-944899468ae0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:20.012889 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f39fa3-5bb0-42db-85a6-944899468ae0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:20.019245 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 277/543] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:14:19 2022] GET /identity/v3/role_assignments?user.id=dfea2ee0c9e84b48ac9bafe3173d937a&scope.project.id=d18a5117a03b4f21ab3290a15680de66&role.id=faf1138f11d34abe8a862f1a1f7c140a => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:20.047393 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329bd881-7a0f-49a1-b58f-47a80ba6a28f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:20.084447 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f098f-a5f7-4a86-a034-aa03d3296024 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:20.084786 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f098f-a5f7-4a86-a034-aa03d3296024 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:20.084991 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f098f-a5f7-4a86-a034-aa03d3296024 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:20.086533 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 278/544] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:14:20.091675 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b171a-9e5f-4ff5-9bb2-f73ca9291276 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:20.092176 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b171a-9e5f-4ff5-9bb2-f73ca9291276 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:20.092675 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b171a-9e5f-4ff5-9bb2-f73ca9291276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:20.112803 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-023b171a-9e5f-4ff5-9bb2-f73ca9291276 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:20.114508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-023b171a-9e5f-4ff5-9bb2-f73ca9291276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:20.176642 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329bd881-7a0f-49a1-b58f-47a80ba6a28f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:20.243304 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 279/545] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:20.246678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e604e2-abe3-46cf-b40a-113acc8db8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:20.247576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e604e2-abe3-46cf-b40a-113acc8db8d9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:20.248053 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e604e2-abe3-46cf-b40a-113acc8db8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:20.248519 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e604e2-abe3-46cf-b40a-113acc8db8d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:20.260711 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e604e2-abe3-46cf-b40a-113acc8db8d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:20.270730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-b0e604e2-abe3-46cf-b40a-113acc8db8d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:20.270730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:20.270730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:20.270730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:20.270730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:20.270730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:20.270730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:20.270730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:20.270730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:20.270730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:20.270730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:20.270730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:20.270730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:20.270730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:20.270730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:20.270730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:20.271613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:20.271613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:20.271613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:20.271613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:20.271613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:20.271613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:20.271613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:20.271613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:20.271613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:20.271613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:20.271613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:20.271613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:20.271613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:20.271613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:20.271613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:20.271613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:20.272410 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:20.272410 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:20.272410 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:20.272410 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:20.272410 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:20.272410 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:20.272410 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:20.272410 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:20.272410 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:20.272410 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:20.272410 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:20.272410 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:20.275802 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 267/546] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Sep 2 09:14:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:20.280453 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b171a-9e5f-4ff5-9bb2-f73ca9291276 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:20.342732 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:20.437340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b171a-9e5f-4ff5-9bb2-f73ca9291276 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:20.474962 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b311e-b5af-418f-b896-43336d75a4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:20.475573 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b311e-b5af-418f-b896-43336d75a4c1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:20.475854 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b311e-b5af-418f-b896-43336d75a4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:20.476640 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328b311e-b5af-418f-b896-43336d75a4c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:20.477309 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b311e-b5af-418f-b896-43336d75a4c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:20.477617 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b311e-b5af-418f-b896-43336d75a4c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:20.479005 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328b311e-b5af-418f-b896-43336d75a4c1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:20.487403 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 280/547] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Sep 2 09:14:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:20.493081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e604e2-abe3-46cf-b40a-113acc8db8d9 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:20.514798 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:20.649974 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e604e2-abe3-46cf-b40a-113acc8db8d9 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:20.689395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b311e-b5af-418f-b896-43336d75a4c1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:20.700931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77494ab2-14bd-4b3a-b0c6-76f40c0f6164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:20.701576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77494ab2-14bd-4b3a-b0c6-76f40c0f6164 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:20.701862 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77494ab2-14bd-4b3a-b0c6-76f40c0f6164 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:20.702142 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77494ab2-14bd-4b3a-b0c6-76f40c0f6164 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:20.718453 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77494ab2-14bd-4b3a-b0c6-76f40c0f6164 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:20.719945 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77494ab2-14bd-4b3a-b0c6-76f40c0f6164 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:20.744981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-77494ab2-14bd-4b3a-b0c6-76f40c0f6164 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:20.744981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:20.744981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:20.744981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:20.744981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:20.744981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:20.744981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:20.744981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:20.744981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:20.744981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:20.744981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:20.744981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:20.744981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:20.744981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:20.744981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:20.744981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:20.746056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:20.746056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:20.746056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:20.746056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:20.746056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:20.746056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:20.746056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:20.746056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:20.746056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:20.746056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:20.746056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:20.746056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:20.746056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:20.746056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:20.746056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:20.746056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:20.746056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:20.747548 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:20.747548 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:20.747548 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:20.747548 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 268/548] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:20.752346 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77494ab2-14bd-4b3a-b0c6-76f40c0f6164 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:20.804155 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328b311e-b5af-418f-b896-43336d75a4c1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:20.807015 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:20.842072 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c8644-14d4-4dff-b8a6-f64889f6cb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:20.842072 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c8644-14d4-4dff-b8a6-f64889f6cb25 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:20.842072 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c8644-14d4-4dff-b8a6-f64889f6cb25 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:20.842072 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c8644-14d4-4dff-b8a6-f64889f6cb25 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:20.854904 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113c8644-14d4-4dff-b8a6-f64889f6cb25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:20.859903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-113c8644-14d4-4dff-b8a6-f64889f6cb25 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:20.859903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:20.859903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:20.859903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:20.859903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:20.859903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:20.859903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:20.859903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:20.859903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:20.859903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:20.859903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:20.859903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:20.859903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:20.859903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:20.859903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:20.859903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:20.860971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:20.860971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:20.860971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:20.860971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:20.860971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:20.860971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:20.860971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:20.860971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:20.860971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:20.860971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:20.860971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:20.860971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:20.860971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:20.860971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:20.860971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:20.860971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:20.861796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:20.861796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:20.861796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:20.861796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:20.861796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:20.861796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:20.861796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:20.861796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:20.861796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:20.861796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:20.861796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:20.861796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:20.865793 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 281/549] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Sep 2 09:14:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:20.867146 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c8644-14d4-4dff-b8a6-f64889f6cb25 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:20.921217 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:20.981832 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c8644-14d4-4dff-b8a6-f64889f6cb25 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:21.007215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77494ab2-14bd-4b3a-b0c6-76f40c0f6164 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:21.049191 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bae185-9799-49ff-8cc9-879305d32c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:21.050030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bae185-9799-49ff-8cc9-879305d32c83 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:21.050313 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bae185-9799-49ff-8cc9-879305d32c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:21.050583 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bae185-9799-49ff-8cc9-879305d32c83 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:21.051289 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bae185-9799-49ff-8cc9-879305d32c83 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:21.051593 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bae185-9799-49ff-8cc9-879305d32c83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:21.052957 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bae185-9799-49ff-8cc9-879305d32c83 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:21.076716 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 282/550] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Sep 2 09:14:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:14:21.078427 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2bd16-3930-453b-b41e-2493b463042b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:21.079075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2bd16-3930-453b-b41e-2493b463042b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:21.079359 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2bd16-3930-453b-b41e-2493b463042b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:21.079636 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2bd16-3930-453b-b41e-2493b463042b None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:21.080344 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2bd16-3930-453b-b41e-2493b463042b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:21.080653 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2bd16-3930-453b-b41e-2493b463042b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:21.082022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2bd16-3930-453b-b41e-2493b463042b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:21.083950 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bae185-9799-49ff-8cc9-879305d32c83 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:21.111132 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 269/551] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:21.116832 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2bd16-3930-453b-b41e-2493b463042b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:21.135087 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:21.211324 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bae185-9799-49ff-8cc9-879305d32c83 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:21.249221 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650481a6-c37f-4537-a9e4-5aeb3ca0c656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:21.249957 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650481a6-c37f-4537-a9e4-5aeb3ca0c656 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:21.250248 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650481a6-c37f-4537-a9e4-5aeb3ca0c656 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:21.250526 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650481a6-c37f-4537-a9e4-5aeb3ca0c656 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:21.262623 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650481a6-c37f-4537-a9e4-5aeb3ca0c656 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:21.264536 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650481a6-c37f-4537-a9e4-5aeb3ca0c656 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:21.290040 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2bd16-3930-453b-b41e-2493b463042b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:21.305876 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-650481a6-c37f-4537-a9e4-5aeb3ca0c656 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:21.305876 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:21.305876 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:21.305876 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:21.305876 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:21.305876 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:21.305876 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:21.305876 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:21.305876 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:21.305876 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:21.305876 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:21.305876 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:21.305876 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:21.305876 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:21.305876 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:21.305876 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:21.306853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:21.306853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:21.306853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:21.306853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:21.306853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:21.306853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:21.306853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:21.306853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:21.306853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:21.306853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:21.306853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:21.306853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:21.306853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:21.306853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:21.306853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:21.306853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:21.306853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:21.307729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:21.307729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:21.307729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:21.309406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 283/552] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:14:21.312506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650481a6-c37f-4537-a9e4-5aeb3ca0c656 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:21.331145 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:21.369297 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae61f2d-4528-41b7-8dca-8b8eba02a86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:21.369942 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae61f2d-4528-41b7-8dca-8b8eba02a86a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:21.370226 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae61f2d-4528-41b7-8dca-8b8eba02a86a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:21.370505 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae61f2d-4528-41b7-8dca-8b8eba02a86a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:21.395445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae61f2d-4528-41b7-8dca-8b8eba02a86a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:21.396916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae61f2d-4528-41b7-8dca-8b8eba02a86a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:21.408449 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:21.416760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-8ae61f2d-4528-41b7-8dca-8b8eba02a86a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 02 09:14:21.416760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:21.416760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:21.416760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:21.416760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:21.416760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:21.416760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:21.416760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:21.416760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:21.416760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:21.416760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:21.416760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:21.416760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:14:21.416760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:14:21.416760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:14:21.416760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:14:21.423416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:21.423416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:21.423416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:14:21.423416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:21.423416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:14:21.423416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:21.423416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:21.423416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:21.423416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:21.423416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:21.423416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:21.423416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:21.423416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:21.423416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:21.423416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:21.423416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:21.423416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:21.423416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:21.424595 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:21.424595 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:21.424595 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:14:21.424595 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:14:21.424595 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:14:21.424595 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:14:21.424595 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:14:21.424595 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:14:21.424595 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 02 09:14:21.424595 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:21.431839 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 270/553] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:14:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:21.433014 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae61f2d-4528-41b7-8dca-8b8eba02a86a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:21.448061 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650481a6-c37f-4537-a9e4-5aeb3ca0c656 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:21.494897 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dca8a2-0e3b-4561-a138-9a9edc077cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:21.496001 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dca8a2-0e3b-4561-a138-9a9edc077cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:21.496461 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dca8a2-0e3b-4561-a138-9a9edc077cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:21.497157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dca8a2-0e3b-4561-a138-9a9edc077cc4 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:21.498424 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dca8a2-0e3b-4561-a138-9a9edc077cc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:21.499186 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dca8a2-0e3b-4561-a138-9a9edc077cc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:21.501312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dca8a2-0e3b-4561-a138-9a9edc077cc4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:21.517092 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 284/554] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:21.518755 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac4108-cac6-4b06-9dea-0936d721e52b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:21.519188 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac4108-cac6-4b06-9dea-0936d721e52b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:21.519607 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ac4108-cac6-4b06-9dea-0936d721e52b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:21.521184 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 285/555] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:21.522378 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac4108-cac6-4b06-9dea-0936d721e52b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:21.573814 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae61f2d-4528-41b7-8dca-8b8eba02a86a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:21.614122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928079b-597c-4a6b-9527-46ca60d9b329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:21.614767 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928079b-597c-4a6b-9527-46ca60d9b329 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:21.615057 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928079b-597c-4a6b-9527-46ca60d9b329 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:21.615334 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928079b-597c-4a6b-9527-46ca60d9b329 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:21.616070 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3928079b-597c-4a6b-9527-46ca60d9b329 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:21.616379 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3928079b-597c-4a6b-9527-46ca60d9b329 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:21.618402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3928079b-597c-4a6b-9527-46ca60d9b329 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:21.646127 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 271/556] 10.222.0.57 () {56 vars in 1135 bytes} [Fri Sep 2 09:14:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:14:21.647560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73480458-497a-4962-8321-296242456f68 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:21.647848 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73480458-497a-4962-8321-296242456f68 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:21.648701 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73480458-497a-4962-8321-296242456f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:21.660815 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ac4108-cac6-4b06-9dea-0936d721e52b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:21.673436 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-73480458-497a-4962-8321-296242456f68 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:21.674987 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-73480458-497a-4962-8321-296242456f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:21.697840 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71f186-43ee-4543-b64f-aa1c3ee0e2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:21.699077 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71f186-43ee-4543-b64f-aa1c3ee0e2a9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:21.699435 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71f186-43ee-4543-b64f-aa1c3ee0e2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:21.699717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71f186-43ee-4543-b64f-aa1c3ee0e2a9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:21.703707 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:21.708978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71f186-43ee-4543-b64f-aa1c3ee0e2a9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:21.711005 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b71f186-43ee-4543-b64f-aa1c3ee0e2a9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:21.715162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:21.722685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-5b71f186-43ee-4543-b64f-aa1c3ee0e2a9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 02 09:14:21.722685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:21.722685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:21.722685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:21.722685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:21.722685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:21.722685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:21.722685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:21.722685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:21.722685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:21.722685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:21.722685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:21.722685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:14:21.722685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:14:21.722685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:14:21.722685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:14:21.724116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:21.724116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:21.724116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:14:21.724116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:21.724116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:14:21.724116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:21.724116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:21.724116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:21.724116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:21.724116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:21.724116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:21.724116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:21.724116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:21.724116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:21.724116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:21.724116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:21.724116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:21.724116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:21.725327 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:21.725327 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:21.725327 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:14:21.725327 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:14:21.725327 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:14:21.725327 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:14:21.725327 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:14:21.725327 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:14:21.725327 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 02 09:14:21.725327 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:21.729070 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 286/557] 10.222.0.57 () {56 vars in 1093 bytes} [Fri Sep 2 09:14:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:14:21.730005 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71f186-43ee-4543-b64f-aa1c3ee0e2a9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:21.755920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 272/558] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:21.757176 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c19902-8b18-4f8d-b122-b459e7b89122 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:21.757524 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c19902-8b18-4f8d-b122-b459e7b89122 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:21.757801 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c19902-8b18-4f8d-b122-b459e7b89122 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:21.759249 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 273/559] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:21.759815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c19902-8b18-4f8d-b122-b459e7b89122 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:21.872583 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c19902-8b18-4f8d-b122-b459e7b89122 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:21.881142 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71f186-43ee-4543-b64f-aa1c3ee0e2a9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:21.910647 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a400d-08a0-4956-a23a-fcf22c603240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:21.911366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a400d-08a0-4956-a23a-fcf22c603240 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:21.912092 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a400d-08a0-4956-a23a-fcf22c603240 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:21.912092 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a400d-08a0-4956-a23a-fcf22c603240 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:21.912801 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a400d-08a0-4956-a23a-fcf22c603240 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:21.913110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a400d-08a0-4956-a23a-fcf22c603240 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:21.914770 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a400d-08a0-4956-a23a-fcf22c603240 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:21.930682 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4127b6-a634-4c73-8b7a-15ef4de55b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:21.931934 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4127b6-a634-4c73-8b7a-15ef4de55b06 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:21.932399 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4127b6-a634-4c73-8b7a-15ef4de55b06 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:21.932848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4127b6-a634-4c73-8b7a-15ef4de55b06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:21.938069 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 274/560] 10.222.0.57 () {56 vars in 1133 bytes} [Fri Sep 2 09:14:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:21.939643 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19500f46-a2d4-4f83-9fa6-d4e92ecbe210 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:21.939998 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19500f46-a2d4-4f83-9fa6-d4e92ecbe210 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:21.940345 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19500f46-a2d4-4f83-9fa6-d4e92ecbe210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:21.960163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-19500f46-a2d4-4f83-9fa6-d4e92ecbe210 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:21.962135 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-19500f46-a2d4-4f83-9fa6-d4e92ecbe210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:21.984933 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:22.014525 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:22.024731 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4127b6-a634-4c73-8b7a-15ef4de55b06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:22.028086 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4127b6-a634-4c73-8b7a-15ef4de55b06 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:22.041557 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 275/561] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:22.042302 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19500f46-a2d4-4f83-9fa6-d4e92ecbe210 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:22.147139 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 287/562] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:14:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:14:22.148499 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4127b6-a634-4c73-8b7a-15ef4de55b06 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:22.163017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19500f46-a2d4-4f83-9fa6-d4e92ecbe210 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:22.201086 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b91811-e1d5-4819-86c2-dd7989e2e8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:22.201950 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b91811-e1d5-4819-86c2-dd7989e2e8a3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:22.202407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b91811-e1d5-4819-86c2-dd7989e2e8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:22.202865 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b91811-e1d5-4819-86c2-dd7989e2e8a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:22.215192 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b91811-e1d5-4819-86c2-dd7989e2e8a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:22.217055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b91811-e1d5-4819-86c2-dd7989e2e8a3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:22.236339 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-38b91811-e1d5-4819-86c2-dd7989e2e8a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:22.236339 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:22.236339 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:22.236339 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:22.236339 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:22.236339 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:22.236339 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:22.236339 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:22.236339 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:22.236339 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:22.236339 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:22.236339 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:22.236339 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:22.236339 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:22.236339 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:22.236339 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:22.237469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:22.237469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:22.237469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:22.237469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:22.237469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:22.237469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:22.237469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:22.237469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:22.237469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:22.237469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:22.237469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:22.237469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:22.237469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:22.237469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:22.237469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:22.237469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:22.237469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:22.238439 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:22.238439 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:22.238439 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:22.238439 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 276/563] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:22.238760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a90f9-eeaa-43d9-8a76-b6721f8bdad7 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:22.239032 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a90f9-eeaa-43d9-8a76-b6721f8bdad7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:22.239305 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a90f9-eeaa-43d9-8a76-b6721f8bdad7 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:22.240691 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 277/564] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:22.245773 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400a90f9-eeaa-43d9-8a76-b6721f8bdad7 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:22.252727 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:22.257028 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:22.259414 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4127b6-a634-4c73-8b7a-15ef4de55b06 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:22.296809 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bbe9e-187b-49b0-a0bd-8830cb3d7566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:22.297552 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bbe9e-187b-49b0-a0bd-8830cb3d7566 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:22.297766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bbe9e-187b-49b0-a0bd-8830cb3d7566 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:22.298038 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bbe9e-187b-49b0-a0bd-8830cb3d7566 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:22.309045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968bbe9e-187b-49b0-a0bd-8830cb3d7566 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:22.313767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-968bbe9e-187b-49b0-a0bd-8830cb3d7566 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:22.313767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:22.313767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:22.313767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:22.313767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:22.313767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:22.313767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:22.313767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:22.313767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:22.313767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:22.313767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:22.313767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:22.313767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:22.313767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:22.313767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:22.313767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:22.314827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:22.314827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:22.314827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:22.314827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:22.314827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:22.314827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:22.314827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:22.314827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:22.314827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:22.314827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:22.314827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:22.314827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:22.314827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:22.314827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:22.314827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:22.314827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:22.315670 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:22.315670 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:22.315670 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:22.315670 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:22.315670 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:22.315670 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:22.315670 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:22.315670 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:22.315670 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:22.315670 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:22.315670 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:22.315670 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:22.315670 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 288/565] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Sep 2 09:14:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:22.316349 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bbe9e-187b-49b0-a0bd-8830cb3d7566 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:22.352829 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400a90f9-eeaa-43d9-8a76-b6721f8bdad7 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:22.385258 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d77fbc-7977-405c-9dc6-815c34eddec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:22.385887 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d77fbc-7977-405c-9dc6-815c34eddec8 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:22.386164 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d77fbc-7977-405c-9dc6-815c34eddec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:22.386440 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d77fbc-7977-405c-9dc6-815c34eddec8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:22.397576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d77fbc-7977-405c-9dc6-815c34eddec8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:22.399516 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d77fbc-7977-405c-9dc6-815c34eddec8 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:22.414426 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d4d77fbc-7977-405c-9dc6-815c34eddec8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:22.414426 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:22.414426 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:22.414426 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:22.414426 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:22.414426 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:22.414426 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:22.414426 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:22.414426 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:22.414426 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:22.414426 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:22.414426 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:22.414426 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:22.414426 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:22.414426 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:22.414426 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:22.415600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:22.415600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:22.415600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:22.415600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:22.415600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:22.415600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:22.415600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:22.415600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:22.415600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:22.415600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:22.415600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:22.415600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:22.415600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:22.415600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:22.415600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:22.415600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:22.415600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:22.416748 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:22.416748 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:22.416748 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:22.416748 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 278/566] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:22.416748 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2cde7-7c5b-4253-b98a-7f0485baffa1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:22.417096 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2cde7-7c5b-4253-b98a-7f0485baffa1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:22.417179 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2cde7-7c5b-4253-b98a-7f0485baffa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:22.422308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bbe9e-187b-49b0-a0bd-8830cb3d7566 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:22.434764 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-50b2cde7-7c5b-4253-b98a-7f0485baffa1 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:22.435971 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-50b2cde7-7c5b-4253-b98a-7f0485baffa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:22.455720 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ffa95f-c716-46a9-abf1-979591c629c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:22.456324 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ffa95f-c716-46a9-abf1-979591c629c6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:22.456604 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ffa95f-c716-46a9-abf1-979591c629c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:22.456874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ffa95f-c716-46a9-abf1-979591c629c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:22.457517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ffa95f-c716-46a9-abf1-979591c629c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:22.457819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ffa95f-c716-46a9-abf1-979591c629c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:22.459218 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ffa95f-c716-46a9-abf1-979591c629c6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:22.471238 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 289/567] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:22.472025 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ffa95f-c716-46a9-abf1-979591c629c6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:22.489501 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:22.492460 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:22.503195 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 279/568] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:22.504430 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2cde7-7c5b-4253-b98a-7f0485baffa1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:22.575837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ffa95f-c716-46a9-abf1-979591c629c6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:22.604882 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2cde7-7c5b-4253-b98a-7f0485baffa1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:22.613294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456703c-fbe9-4ac3-afe7-8dc52f804e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:22.614250 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456703c-fbe9-4ac3-afe7-8dc52f804e5d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:22.614681 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456703c-fbe9-4ac3-afe7-8dc52f804e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:22.615128 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456703c-fbe9-4ac3-afe7-8dc52f804e5d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:22.625944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c456703c-fbe9-4ac3-afe7-8dc52f804e5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:22.631395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-c456703c-fbe9-4ac3-afe7-8dc52f804e5d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:22.631395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:22.631395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:22.631395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:22.631395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:22.631395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:22.631395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:22.631395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:22.631395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:22.631395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:22.631395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:22.631395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:22.631395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:22.631395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:22.631395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:22.631395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:22.633011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:22.633011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:22.633011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:22.633011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:22.633011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:22.633011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:22.633011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:22.633011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:22.633011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:22.633011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:22.633011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:22.633011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:22.633011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:22.633011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:22.633011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:22.633011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:22.634048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:22.634048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:22.634048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:22.634048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:22.634048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:22.634048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:22.634048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:22.634048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:22.634048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:22.634048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:22.634048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:22.634048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:22.636105 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 290/569] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Sep 2 09:14:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:22.637070 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456703c-fbe9-4ac3-afe7-8dc52f804e5d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:22.642338 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3b66d-3dbe-46c0-9521-7b77ca8e8a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:22.642975 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3b66d-3dbe-46c0-9521-7b77ca8e8a5f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:22.643255 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3b66d-3dbe-46c0-9521-7b77ca8e8a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:22.643523 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3b66d-3dbe-46c0-9521-7b77ca8e8a5f None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:22.644186 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b3b66d-3dbe-46c0-9521-7b77ca8e8a5f None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:22.644487 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b3b66d-3dbe-46c0-9521-7b77ca8e8a5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:22.645839 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b3b66d-3dbe-46c0-9521-7b77ca8e8a5f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:22.655011 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 280/570] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Sep 2 09:14:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:22.655730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3b66d-3dbe-46c0-9521-7b77ca8e8a5f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:22.708804 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:22.725876 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:22.751376 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456703c-fbe9-4ac3-afe7-8dc52f804e5d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:22.757969 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3b66d-3dbe-46c0-9521-7b77ca8e8a5f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:22.789789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f88c965-7ff4-4870-a229-97e7c8763a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:22.790389 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88c965-7ff4-4870-a229-97e7c8763a12 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:22.790670 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88c965-7ff4-4870-a229-97e7c8763a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:22.790964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f88c965-7ff4-4870-a229-97e7c8763a12 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:22.791596 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f88c965-7ff4-4870-a229-97e7c8763a12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:22.791894 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f88c965-7ff4-4870-a229-97e7c8763a12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:22.792613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce94485-21ae-4861-9ff6-5020e95c5f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:22.793235 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f88c965-7ff4-4870-a229-97e7c8763a12 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:22.793326 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce94485-21ae-4861-9ff6-5020e95c5f0f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:22.793511 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce94485-21ae-4861-9ff6-5020e95c5f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:22.793783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce94485-21ae-4861-9ff6-5020e95c5f0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:22.800367 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce94485-21ae-4861-9ff6-5020e95c5f0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:22.801772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce94485-21ae-4861-9ff6-5020e95c5f0f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:22.804109 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 291/571] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:22.804891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f88c965-7ff4-4870-a229-97e7c8763a12 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:22.814485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-6ce94485-21ae-4861-9ff6-5020e95c5f0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:22.814485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:22.814485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:22.814485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:22.814485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:22.814485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:22.814485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:22.814485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:22.814485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:22.814485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:22.814485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:22.814485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:22.814485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:14:22.814485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:14:22.814485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:14:22.815422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:14:22.815422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:22.815422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:22.815422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:22.815422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:22.815422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:22.815422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:22.815422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:22.815422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:22.815422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:22.815422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:22.815422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:22.815422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:22.815422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:22.815422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:22.815422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:22.815422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:22.816305 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:14:22.816305 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:14:22.816305 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:14:22.816305 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:14:22.816305 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:14:22.816305 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:14:22.816305 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:22.816305 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:22.816305 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 281/572] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:22.817023 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce94485-21ae-4861-9ff6-5020e95c5f0f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:22.911601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f88c965-7ff4-4870-a229-97e7c8763a12 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:22.920544 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce94485-21ae-4861-9ff6-5020e95c5f0f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:22.932761 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:22.943686 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc05755-0180-4db8-812d-1355e9664529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:22.944288 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc05755-0180-4db8-812d-1355e9664529 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:22.944563 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc05755-0180-4db8-812d-1355e9664529 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:22.945381 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc05755-0180-4db8-812d-1355e9664529 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:22.961760 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc05755-0180-4db8-812d-1355e9664529 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:22.961760 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc05755-0180-4db8-812d-1355e9664529 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:22.961760 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc05755-0180-4db8-812d-1355e9664529 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:22.961760 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:22.962060 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b20aa7-90d6-4b76-be26-c52e95bc9010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:22.962060 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b20aa7-90d6-4b76-be26-c52e95bc9010 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:22.962060 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b20aa7-90d6-4b76-be26-c52e95bc9010 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:22.962060 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b20aa7-90d6-4b76-be26-c52e95bc9010 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:22.962331 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 292/573] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Sep 2 09:14:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:22.962331 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc05755-0180-4db8-812d-1355e9664529 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:22.967161 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b20aa7-90d6-4b76-be26-c52e95bc9010 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:22.968610 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b20aa7-90d6-4b76-be26-c52e95bc9010 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:22.983998 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-e9b20aa7-90d6-4b76-be26-c52e95bc9010 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:22.983998 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:22.983998 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:22.983998 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:22.983998 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:22.983998 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:22.983998 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:22.983998 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:22.983998 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:22.983998 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:22.983998 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:22.983998 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:22.983998 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:22.983998 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:22.983998 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:22.983998 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:22.985178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:22.985178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:22.985178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:22.985178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:22.985178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:22.985178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:22.985178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:22.985178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:22.985178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:22.985178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:22.985178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:22.985178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:22.985178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:22.985178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:22.985178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:22.985178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:22.985178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:22.986306 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:22.986306 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:22.986306 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:22.986306 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 282/574] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:22.986306 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b20aa7-90d6-4b76-be26-c52e95bc9010 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:23.060883 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc05755-0180-4db8-812d-1355e9664529 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:23.084221 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b20aa7-90d6-4b76-be26-c52e95bc9010 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:23.094474 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ab70c-66b5-45e8-899b-39e0067857aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:23.095117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ab70c-66b5-45e8-899b-39e0067857aa None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:23.095401 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ab70c-66b5-45e8-899b-39e0067857aa None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:23.095674 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ab70c-66b5-45e8-899b-39e0067857aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:23.105437 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ab70c-66b5-45e8-899b-39e0067857aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:23.110189 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-1b4ab70c-66b5-45e8-899b-39e0067857aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:23.110189 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:23.110189 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:23.110189 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:23.110189 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:23.110189 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:23.110189 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:23.110189 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:23.110189 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:23.110189 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:23.110189 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:23.110189 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:23.110189 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:23.110189 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:23.110189 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:23.110189 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:23.111187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:23.111187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:23.111187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:23.111187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:23.111187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:23.111187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:23.111187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:23.111187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:23.111187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:23.111187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:23.111187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:23.111187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:23.111187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:23.111187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:23.111187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:23.111187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:23.112008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:23.112008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:23.112008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:23.112008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:23.112008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:23.112008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:23.112008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:23.112008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:23.112008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:23.112008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:23.112008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:23.112008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:23.112008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 293/575] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Sep 2 09:14:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:23.112685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ab70c-66b5-45e8-899b-39e0067857aa None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:23.119834 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da897c-44ad-4fdc-9893-03208a3f858b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:23.120458 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da897c-44ad-4fdc-9893-03208a3f858b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:23.121143 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da897c-44ad-4fdc-9893-03208a3f858b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:23.121421 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da897c-44ad-4fdc-9893-03208a3f858b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:23.128713 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6da897c-44ad-4fdc-9893-03208a3f858b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:23.130135 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6da897c-44ad-4fdc-9893-03208a3f858b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:23.142031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-a6da897c-44ad-4fdc-9893-03208a3f858b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:23.142031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:23.142031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:23.142031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:23.142031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:23.142031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:23.142031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:23.142031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:23.142031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:23.142031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:23.142031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:23.142031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:23.142031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:14:23.142031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:14:23.142031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:14:23.143191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:14:23.143191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:23.143191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:23.143191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:23.143191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:23.143191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:23.143191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:23.143191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:23.143191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:23.143191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:23.143191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:23.143191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:23.143191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:23.143191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:23.143191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:23.143191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:23.143191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:23.144404 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:14:23.144404 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:14:23.144404 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:14:23.144404 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:14:23.144404 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:14:23.144404 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:14:23.144404 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:23.144404 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:23.144404 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 283/576] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:23.144404 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da897c-44ad-4fdc-9893-03208a3f858b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:23.168136 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:23.176894 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:23.221929 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ab70c-66b5-45e8-899b-39e0067857aa None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:23.243100 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da897c-44ad-4fdc-9893-03208a3f858b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:23.254607 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ecabc8-a8f7-4df5-94b2-f2c97f619224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:23.255239 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ecabc8-a8f7-4df5-94b2-f2c97f619224 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:23.255513 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ecabc8-a8f7-4df5-94b2-f2c97f619224 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:23.255780 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ecabc8-a8f7-4df5-94b2-f2c97f619224 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:23.256468 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ecabc8-a8f7-4df5-94b2-f2c97f619224 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:23.256767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ecabc8-a8f7-4df5-94b2-f2c97f619224 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:23.258791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ecabc8-a8f7-4df5-94b2-f2c97f619224 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:23.275600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2302cb-7c89-4617-8f93-342877f2d20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:23.275743 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 294/577] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:14:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:23.276252 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2302cb-7c89-4617-8f93-342877f2d20a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:23.276505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ecabc8-a8f7-4df5-94b2-f2c97f619224 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:23.276608 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2302cb-7c89-4617-8f93-342877f2d20a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:23.276855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2302cb-7c89-4617-8f93-342877f2d20a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:23.289454 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2302cb-7c89-4617-8f93-342877f2d20a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:23.291001 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2302cb-7c89-4617-8f93-342877f2d20a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:23.305807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-7f2302cb-7c89-4617-8f93-342877f2d20a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:23.305807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:23.305807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:23.305807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:23.305807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:23.305807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:23.305807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:23.305807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:23.305807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:23.305807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:23.305807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:23.305807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:23.305807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:23.305807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:23.305807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:23.305807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:23.306765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:23.306765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:23.306765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:23.306765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:23.306765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:23.306765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:23.306765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:23.306765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:23.306765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:23.306765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:23.306765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:23.306765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:23.306765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:23.306765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:23.306765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:23.306765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:23.306765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:23.307678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:23.307678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:23.307678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:23.307678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 284/578] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:23.307902 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2302cb-7c89-4617-8f93-342877f2d20a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:23.378590 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ecabc8-a8f7-4df5-94b2-f2c97f619224 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:23.384129 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:23.395122 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:23.406307 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2302cb-7c89-4617-8f93-342877f2d20a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:23.411655 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e99841-ff85-40eb-b71e-e85d5eaffd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:23.412264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e99841-ff85-40eb-b71e-e85d5eaffd6f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:23.412546 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e99841-ff85-40eb-b71e-e85d5eaffd6f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:23.412818 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e99841-ff85-40eb-b71e-e85d5eaffd6f None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:23.413975 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e99841-ff85-40eb-b71e-e85d5eaffd6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:23.414280 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e99841-ff85-40eb-b71e-e85d5eaffd6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:23.415657 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e99841-ff85-40eb-b71e-e85d5eaffd6f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:23.426757 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 295/579] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:23.427545 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e99841-ff85-40eb-b71e-e85d5eaffd6f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:23.442579 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b84bee-d135-48bd-a34d-2d7c1b4024cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:23.443228 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b84bee-d135-48bd-a34d-2d7c1b4024cc None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:23.443509 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b84bee-d135-48bd-a34d-2d7c1b4024cc None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:23.443788 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b84bee-d135-48bd-a34d-2d7c1b4024cc None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:23.444448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b84bee-d135-48bd-a34d-2d7c1b4024cc None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:23.444753 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b84bee-d135-48bd-a34d-2d7c1b4024cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:23.446102 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b84bee-d135-48bd-a34d-2d7c1b4024cc None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:23.454181 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 285/580] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Sep 2 09:14:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:23.454967 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b84bee-d135-48bd-a34d-2d7c1b4024cc None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:23.535005 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e99841-ff85-40eb-b71e-e85d5eaffd6f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:23.558634 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b84bee-d135-48bd-a34d-2d7c1b4024cc None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:23.567493 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb11863-af04-405b-aaad-23344e0cf0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:23.568094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb11863-af04-405b-aaad-23344e0cf0ae None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:23.568368 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb11863-af04-405b-aaad-23344e0cf0ae None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:23.568636 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb11863-af04-405b-aaad-23344e0cf0ae None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:23.569323 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb11863-af04-405b-aaad-23344e0cf0ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:23.569619 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb11863-af04-405b-aaad-23344e0cf0ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:23.571648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb11863-af04-405b-aaad-23344e0cf0ae None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:23.583660 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 296/581] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:14:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:23.584450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb11863-af04-405b-aaad-23344e0cf0ae None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:23.595499 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c34119-5965-4321-9a03-5085ea515a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:23.596159 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c34119-5965-4321-9a03-5085ea515a6e None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:23.596446 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c34119-5965-4321-9a03-5085ea515a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:23.598783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c34119-5965-4321-9a03-5085ea515a6e None admin] PATH_INFO: `/v3/projects/d18a5117a03b4f21ab3290a15680de66/users/1da355c8be2d496bb08feb0fe74e04b9/roles/faf1138f11d34abe8a862f1a1f7c140a` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:23.605212 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:23.611456 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c34119-5965-4321-9a03-5085ea515a6e None admin] RBAC: Authorizing `identity:create_grant(project_id=d18a5117a03b4f21ab3290a15680de66, user_id=1da355c8be2d496bb08feb0fe74e04b9, role_id=faf1138f11d34abe8a862f1a1f7c140a)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:23.613061 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c34119-5965-4321-9a03-5085ea515a6e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:23.628411 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:23.684615 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb11863-af04-405b-aaad-23344e0cf0ae None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:23.709366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 286/582] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Sep 2 09:14:23 2022] PUT /identity/v3/projects/d18a5117a03b4f21ab3290a15680de66/users/1da355c8be2d496bb08feb0fe74e04b9/roles/faf1138f11d34abe8a862f1a1f7c140a => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:23.710184 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c34119-5965-4321-9a03-5085ea515a6e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:23.720162 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432e03e-dcb4-4e16-ace7-e02963fc6f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:23.720795 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432e03e-dcb4-4e16-ace7-e02963fc6f92 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:23.721076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432e03e-dcb4-4e16-ace7-e02963fc6f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:23.721349 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432e03e-dcb4-4e16-ace7-e02963fc6f92 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:23.722015 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432e03e-dcb4-4e16-ace7-e02963fc6f92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:23.722321 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432e03e-dcb4-4e16-ace7-e02963fc6f92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:23.723800 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432e03e-dcb4-4e16-ace7-e02963fc6f92 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:23.742217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 297/583] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:23.743237 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432e03e-dcb4-4e16-ace7-e02963fc6f92 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:23.835484 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c34119-5965-4321-9a03-5085ea515a6e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:23.859718 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:23.865088 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432e03e-dcb4-4e16-ace7-e02963fc6f92 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:23.878286 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f50c5-14bc-4724-b0b1-97bea41bbdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:23.878996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f50c5-14bc-4724-b0b1-97bea41bbdbe None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:23.879351 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f50c5-14bc-4724-b0b1-97bea41bbdbe None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:23.879633 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f50c5-14bc-4724-b0b1-97bea41bbdbe None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:23.887365 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404f50c5-14bc-4724-b0b1-97bea41bbdbe None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:23.888927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404f50c5-14bc-4724-b0b1-97bea41bbdbe None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:23.901931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-404f50c5-14bc-4724-b0b1-97bea41bbdbe None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 02 09:14:23.901931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:23.901931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:23.901931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:23.901931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:23.901931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:23.901931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:23.901931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:23.901931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:23.901931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:23.901931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:23.901931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:23.901931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:14:23.901931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:14:23.901931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:14:23.901931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:14:23.902926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:23.902926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:23.902926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:14:23.902926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:23.902926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:14:23.902926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:23.902926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:23.902926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:23.902926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:23.902926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:23.902926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:23.902926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:23.902926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:23.902926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:23.902926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:23.902926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:23.902926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:23.902926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:23.904292 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:23.904292 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:23.904292 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:14:23.904292 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:14:23.904292 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:14:23.904292 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:14:23.904292 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:14:23.904292 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:14:23.904292 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 02 09:14:23.904292 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:23.904292 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 287/584] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:14:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:23.904292 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f50c5-14bc-4724-b0b1-97bea41bbdbe None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:23.912772 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef4bc3-114e-4f13-8b36-af8843b22968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:23.913486 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef4bc3-114e-4f13-8b36-af8843b22968 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:23.914262 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef4bc3-114e-4f13-8b36-af8843b22968 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:23.914538 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef4bc3-114e-4f13-8b36-af8843b22968 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:23.926172 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef4bc3-114e-4f13-8b36-af8843b22968 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:23.927662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef4bc3-114e-4f13-8b36-af8843b22968 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:23.942182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-60ef4bc3-114e-4f13-8b36-af8843b22968 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:23.942182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:23.942182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:23.942182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:23.942182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:23.942182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:23.942182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:23.942182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:23.942182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:23.942182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:23.942182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:23.942182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:23.942182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:23.942182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:23.942182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:23.942182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:23.943151 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:23.943151 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:23.943151 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:23.943151 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:23.943151 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:23.943151 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:23.943151 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:23.943151 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:23.943151 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:23.943151 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:23.943151 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:23.943151 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:23.943151 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:23.943151 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:23.943151 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:23.943151 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:23.943151 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:23.949164 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:23.949164 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:23.949164 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:23.949164 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 298/585] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:23.949164 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef4bc3-114e-4f13-8b36-af8843b22968 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:23.959800 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:24.012180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f50c5-14bc-4724-b0b1-97bea41bbdbe None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:24.049185 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78733fd6-f432-4220-acf2-928255b5ac79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:24.049800 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78733fd6-f432-4220-acf2-928255b5ac79 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:24.050081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78733fd6-f432-4220-acf2-928255b5ac79 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:24.050364 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78733fd6-f432-4220-acf2-928255b5ac79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:24.050829 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef4bc3-114e-4f13-8b36-af8843b22968 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:24.051131 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78733fd6-f432-4220-acf2-928255b5ac79 None admin] RBAC: Adding query filter params (role.id=faf1138f11d34abe8a862f1a1f7c140a, scope.project.id=d18a5117a03b4f21ab3290a15680de66, user.id=30fe233a7d4c4fbea6ede233bd9ff8c8) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:24.051420 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78733fd6-f432-4220-acf2-928255b5ac79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:24.052864 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78733fd6-f432-4220-acf2-928255b5ac79 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:24.059396 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 288/586] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:14:23 2022] GET /identity/v3/role_assignments?user.id=30fe233a7d4c4fbea6ede233bd9ff8c8&scope.project.id=d18a5117a03b4f21ab3290a15680de66&role.id=faf1138f11d34abe8a862f1a1f7c140a => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:24.060106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78733fd6-f432-4220-acf2-928255b5ac79 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:24.102865 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef251c1-5d14-4356-9864-2615403e9b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:24.103910 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef251c1-5d14-4356-9864-2615403e9b1e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:24.104427 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef251c1-5d14-4356-9864-2615403e9b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:24.104954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef251c1-5d14-4356-9864-2615403e9b1e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:24.114831 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef251c1-5d14-4356-9864-2615403e9b1e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:24.116925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef251c1-5d14-4356-9864-2615403e9b1e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:24.130652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-9ef251c1-5d14-4356-9864-2615403e9b1e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 02 09:14:24.130652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:24.130652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:24.130652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:24.130652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:24.130652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:24.130652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:24.130652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:24.130652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:24.130652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:24.130652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:24.130652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:24.130652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:14:24.130652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:14:24.130652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:14:24.130652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:14:24.131700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:24.131700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:24.131700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:14:24.131700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:24.131700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:14:24.131700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:24.131700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:24.131700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:24.131700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:24.131700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:24.131700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:24.131700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:24.131700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:24.131700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:24.131700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:24.131700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:24.131700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:24.131700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:24.133162 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:24.133162 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:24.133162 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:14:24.133162 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:14:24.133162 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:14:24.133162 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:14:24.133162 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:14:24.133162 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:14:24.133162 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 02 09:14:24.133162 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:24.140513 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 299/587] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:14:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:24.142079 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef251c1-5d14-4356-9864-2615403e9b1e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:24.158035 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:24.182576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78733fd6-f432-4220-acf2-928255b5ac79 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:24.197357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:24.225575 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9445b400-733c-4e2b-afef-db59fc766e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:24.226589 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9445b400-733c-4e2b-afef-db59fc766e0e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:24.227163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9445b400-733c-4e2b-afef-db59fc766e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:24.227637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9445b400-733c-4e2b-afef-db59fc766e0e None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:24.228589 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9445b400-733c-4e2b-afef-db59fc766e0e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:24.229128 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9445b400-733c-4e2b-afef-db59fc766e0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:24.230800 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9445b400-733c-4e2b-afef-db59fc766e0e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:24.254394 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 289/588] 10.222.0.57 () {56 vars in 1137 bytes} [Fri Sep 2 09:14:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:24.255787 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9445b400-733c-4e2b-afef-db59fc766e0e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:24.269605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef251c1-5d14-4356-9864-2615403e9b1e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:24.312442 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceaa726-03aa-4b46-b82a-e688dd843ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:24.313054 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceaa726-03aa-4b46-b82a-e688dd843ddb None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:24.313333 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceaa726-03aa-4b46-b82a-e688dd843ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:24.313606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceaa726-03aa-4b46-b82a-e688dd843ddb None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:24.314521 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceaa726-03aa-4b46-b82a-e688dd843ddb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:24.314859 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceaa726-03aa-4b46-b82a-e688dd843ddb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:24.316417 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceaa726-03aa-4b46-b82a-e688dd843ddb None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:24.338514 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 300/589] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:24.338514 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceaa726-03aa-4b46-b82a-e688dd843ddb None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:24.378283 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9445b400-733c-4e2b-afef-db59fc766e0e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:24.417160 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:24.423802 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e066c8-0d27-491d-b3b2-417b7121c90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:24.424414 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e066c8-0d27-491d-b3b2-417b7121c90a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:24.424697 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e066c8-0d27-491d-b3b2-417b7121c90a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:24.425362 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e066c8-0d27-491d-b3b2-417b7121c90a None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:24.426089 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e066c8-0d27-491d-b3b2-417b7121c90a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:24.426400 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e066c8-0d27-491d-b3b2-417b7121c90a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:24.428213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e066c8-0d27-491d-b3b2-417b7121c90a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:24.454813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 290/590] 10.222.0.57 () {56 vars in 1137 bytes} [Fri Sep 2 09:14:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:24.454813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e066c8-0d27-491d-b3b2-417b7121c90a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:24.456478 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceaa726-03aa-4b46-b82a-e688dd843ddb None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:24.488929 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:24.498127 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e7a026-4139-4425-b1be-9a145f902eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:24.503515 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7a026-4139-4425-b1be-9a145f902eab None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:24.503515 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7a026-4139-4425-b1be-9a145f902eab None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:24.503515 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e7a026-4139-4425-b1be-9a145f902eab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:24.511279 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e7a026-4139-4425-b1be-9a145f902eab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:24.513082 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e7a026-4139-4425-b1be-9a145f902eab None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:24.529264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-48e7a026-4139-4425-b1be-9a145f902eab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:24.529264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:24.529264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:24.529264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:24.529264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:24.529264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:24.529264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:24.529264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:24.529264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:24.529264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:24.529264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:24.529264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:24.529264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:24.529264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:24.529264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:24.529264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:24.530228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:24.530228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:24.530228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:24.530228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:24.530228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:24.530228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:24.530228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:24.530228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:24.530228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:24.530228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:24.530228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:24.530228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:24.530228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:24.530228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:24.530228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:24.530228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:24.530228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:24.531150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:24.531150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:24.531150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:24.537364 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 301/591] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:24.538429 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e7a026-4139-4425-b1be-9a145f902eab None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:24.570134 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e066c8-0d27-491d-b3b2-417b7121c90a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:24.607600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ca84e-edf6-4a0e-aec9-306e00372266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:24.608240 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ca84e-edf6-4a0e-aec9-306e00372266 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:24.608613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ca84e-edf6-4a0e-aec9-306e00372266 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:24.608891 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ca84e-edf6-4a0e-aec9-306e00372266 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:24.616552 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ca84e-edf6-4a0e-aec9-306e00372266 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:24.618003 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ca84e-edf6-4a0e-aec9-306e00372266 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:24.630356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-5e7ca84e-edf6-4a0e-aec9-306e00372266 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 02 09:14:24.630356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:24.630356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:24.630356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:24.630356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:24.630356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:24.630356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:24.630356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:24.630356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:24.630356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:24.630356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:24.630356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:24.630356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:14:24.630356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:14:24.630356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:14:24.630356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:14:24.631541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:24.631541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:24.631541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:14:24.631541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:24.631541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:14:24.631541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:24.631541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:24.631541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:24.631541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:24.631541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:24.631541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:24.631541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:24.631541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:24.631541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:24.631541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:24.631541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:24.631541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:24.631541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:24.632739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:24.632739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:24.632739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:14:24.632739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:14:24.632739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:14:24.632739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:14:24.632739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:14:24.632739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:14:24.632739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 02 09:14:24.632739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:24.635066 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 291/592] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Sep 2 09:14:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:24.635903 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ca84e-edf6-4a0e-aec9-306e00372266 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:24.649446 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e7a026-4139-4425-b1be-9a145f902eab None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:24.673705 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:24.685747 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807dace-4aaa-4da2-a556-fc4783531640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:24.686602 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807dace-4aaa-4da2-a556-fc4783531640 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:24.687060 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807dace-4aaa-4da2-a556-fc4783531640 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:24.687500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807dace-4aaa-4da2-a556-fc4783531640 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:24.700051 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807dace-4aaa-4da2-a556-fc4783531640 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:24.701956 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f807dace-4aaa-4da2-a556-fc4783531640 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:24.721214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-f807dace-4aaa-4da2-a556-fc4783531640 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:24.721214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:24.721214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:24.721214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:24.721214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:24.721214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:24.721214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:24.721214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:24.721214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:24.721214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:24.721214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:24.721214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:24.721214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:24.721214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:24.721214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:24.721214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:24.722410 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:24.722410 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:24.722410 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:24.722410 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:24.722410 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:24.722410 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:24.722410 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:24.722410 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:24.722410 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:24.722410 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:24.722410 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:24.722410 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:24.722410 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:24.722410 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:24.722410 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:24.722410 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:24.722410 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:24.723560 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:24.723560 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:24.723560 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:24.724523 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 302/593] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:24.729712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807dace-4aaa-4da2-a556-fc4783531640 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:24.741673 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:24.755360 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ca84e-edf6-4a0e-aec9-306e00372266 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:24.791905 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871263c-be35-44ce-82d0-d06568f2cdf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:24.792694 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871263c-be35-44ce-82d0-d06568f2cdf4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:24.793072 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871263c-be35-44ce-82d0-d06568f2cdf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:24.793438 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2871263c-be35-44ce-82d0-d06568f2cdf4 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:24.794201 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2871263c-be35-44ce-82d0-d06568f2cdf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:24.794595 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2871263c-be35-44ce-82d0-d06568f2cdf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:24.796313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2871263c-be35-44ce-82d0-d06568f2cdf4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:24.814601 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 292/594] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:24.814601 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871263c-be35-44ce-82d0-d06568f2cdf4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:24.842811 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807dace-4aaa-4da2-a556-fc4783531640 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:24.878862 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50efc8dc-1d9c-4377-94f6-52d875221329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:24.879740 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efc8dc-1d9c-4377-94f6-52d875221329 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:24.880169 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efc8dc-1d9c-4377-94f6-52d875221329 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:24.880588 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efc8dc-1d9c-4377-94f6-52d875221329 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:24.881491 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50efc8dc-1d9c-4377-94f6-52d875221329 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:24.881977 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50efc8dc-1d9c-4377-94f6-52d875221329 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:24.884136 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50efc8dc-1d9c-4377-94f6-52d875221329 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:24.923328 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 303/595] 10.222.0.57 () {56 vars in 1139 bytes} [Fri Sep 2 09:14:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:24.924971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50efc8dc-1d9c-4377-94f6-52d875221329 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:24.936151 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2871263c-be35-44ce-82d0-d06568f2cdf4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:24.957864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:24.984027 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aaa5b3-57e5-402b-b8c9-82dd3e9b0b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:24.984027 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aaa5b3-57e5-402b-b8c9-82dd3e9b0b17 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:24.984027 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aaa5b3-57e5-402b-b8c9-82dd3e9b0b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:24.984027 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aaa5b3-57e5-402b-b8c9-82dd3e9b0b17 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:24.984403 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aaa5b3-57e5-402b-b8c9-82dd3e9b0b17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:24.984403 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aaa5b3-57e5-402b-b8c9-82dd3e9b0b17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:24.984403 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aaa5b3-57e5-402b-b8c9-82dd3e9b0b17 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:24.992995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 293/596] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:24.992995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aaa5b3-57e5-402b-b8c9-82dd3e9b0b17 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:25.007649 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:25.043700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50efc8dc-1d9c-4377-94f6-52d875221329 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:25.089793 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb52e3-913f-41d3-ac18-715328f52f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:25.090437 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb52e3-913f-41d3-ac18-715328f52f5a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:25.090731 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb52e3-913f-41d3-ac18-715328f52f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:25.091012 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb52e3-913f-41d3-ac18-715328f52f5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:25.099582 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb52e3-913f-41d3-ac18-715328f52f5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:25.101715 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb52e3-913f-41d3-ac18-715328f52f5a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:25.116336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-e9fb52e3-913f-41d3-ac18-715328f52f5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:25.116336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:25.116336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:25.116336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:25.116336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:25.116336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:25.116336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:25.116336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:25.116336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:25.116336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:25.116336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:25.116336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:25.116336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:14:25.116336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:14:25.116336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:14:25.117246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:14:25.117246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:25.117246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:25.117246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:25.117246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:25.117246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:25.117246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:25.117246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:25.117246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:25.117246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:25.117246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:25.117246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:25.117246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:25.117246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:25.117246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:25.117246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:25.117246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:25.118121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:14:25.118121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:14:25.118121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:14:25.118121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:14:25.118121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:14:25.118121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:14:25.118121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:25.118121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:25.119057 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aaa5b3-57e5-402b-b8c9-82dd3e9b0b17 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:25.122484 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 304/597] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:25.124190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb52e3-913f-41d3-ac18-715328f52f5a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:25.171011 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c0a73-b850-4f4b-8910-f9ac3c2646d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:25.171861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c0a73-b850-4f4b-8910-f9ac3c2646d2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:25.172522 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c0a73-b850-4f4b-8910-f9ac3c2646d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:25.172927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c0a73-b850-4f4b-8910-f9ac3c2646d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:25.187831 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7c0a73-b850-4f4b-8910-f9ac3c2646d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:25.189990 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7c0a73-b850-4f4b-8910-f9ac3c2646d2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:25.211118 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-7d7c0a73-b850-4f4b-8910-f9ac3c2646d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:25.211118 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:25.211118 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:25.211118 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:25.211118 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:25.211118 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:25.211118 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:25.211118 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:25.211118 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:25.211118 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:25.211118 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:25.211118 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:25.211118 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:25.211118 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:25.211118 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:25.211118 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:25.212080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:25.212080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:25.212080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:25.212080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:25.212080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:25.212080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:25.212080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:25.212080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:25.212080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:25.212080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:25.212080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:25.212080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:25.212080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:25.212080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:25.212080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:25.212080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:25.212080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:25.212959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:25.212959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:25.212959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:25.212959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 294/598] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:25.213938 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c0a73-b850-4f4b-8910-f9ac3c2646d2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:25.254919 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:25.267449 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb52e3-913f-41d3-ac18-715328f52f5a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:25.281489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:25.313969 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998accc2-833b-4567-8027-3d851b28e71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:25.315294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998accc2-833b-4567-8027-3d851b28e71a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:25.315729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998accc2-833b-4567-8027-3d851b28e71a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:25.316437 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998accc2-833b-4567-8027-3d851b28e71a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:25.324800 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7c0a73-b850-4f4b-8910-f9ac3c2646d2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:25.328047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998accc2-833b-4567-8027-3d851b28e71a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:25.330245 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998accc2-833b-4567-8027-3d851b28e71a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:25.342150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-998accc2-833b-4567-8027-3d851b28e71a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:25.342150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:25.342150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:25.342150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:25.342150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:25.342150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:25.342150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:25.342150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:25.342150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:25.342150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:25.342150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:25.342150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:25.342150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:14:25.342150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:14:25.342150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:14:25.343264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:14:25.343264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:25.343264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:25.343264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:25.343264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:25.343264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:25.343264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:25.343264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:25.343264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:25.343264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:25.343264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:25.343264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:25.343264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:25.343264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:25.343264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:25.343264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:25.343264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:25.344357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:14:25.344357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:14:25.344357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:14:25.344357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:14:25.344357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:14:25.344357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:14:25.344357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:25.344357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:25.346069 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 305/599] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:25.347037 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998accc2-833b-4567-8027-3d851b28e71a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:25.373477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda71714-cb40-4b68-be6a-cae68cca561b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:25.374509 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda71714-cb40-4b68-be6a-cae68cca561b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:25.374979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda71714-cb40-4b68-be6a-cae68cca561b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:25.375719 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda71714-cb40-4b68-be6a-cae68cca561b None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:25.376852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda71714-cb40-4b68-be6a-cae68cca561b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:25.377300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda71714-cb40-4b68-be6a-cae68cca561b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:25.379650 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda71714-cb40-4b68-be6a-cae68cca561b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:25.400951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 295/600] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:14:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:25.400951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda71714-cb40-4b68-be6a-cae68cca561b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:25.481473 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998accc2-833b-4567-8027-3d851b28e71a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:25.509112 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda71714-cb40-4b68-be6a-cae68cca561b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:25.519362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084cda36-38d2-43e9-b637-2cb3e73acf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:25.519986 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084cda36-38d2-43e9-b637-2cb3e73acf14 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:25.520271 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084cda36-38d2-43e9-b637-2cb3e73acf14 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:25.520546 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084cda36-38d2-43e9-b637-2cb3e73acf14 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:25.521333 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084cda36-38d2-43e9-b637-2cb3e73acf14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:25.521641 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084cda36-38d2-43e9-b637-2cb3e73acf14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:25.523575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084cda36-38d2-43e9-b637-2cb3e73acf14 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:25.537553 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 306/601] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:25.539068 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084cda36-38d2-43e9-b637-2cb3e73acf14 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:25.540663 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:25.555760 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:25.556192 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ecb303-9dc0-4ae6-9265-0968ec188ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:25.562334 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ecb303-9dc0-4ae6-9265-0968ec188ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:25.562334 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ecb303-9dc0-4ae6-9265-0968ec188ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:25.562334 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ecb303-9dc0-4ae6-9265-0968ec188ae9 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:25.562334 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ecb303-9dc0-4ae6-9265-0968ec188ae9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:25.562334 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ecb303-9dc0-4ae6-9265-0968ec188ae9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:25.562334 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ecb303-9dc0-4ae6-9265-0968ec188ae9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:25.577807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 296/602] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:14:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:25.579693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ecb303-9dc0-4ae6-9265-0968ec188ae9 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:25.658567 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084cda36-38d2-43e9-b637-2cb3e73acf14 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:25.696462 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8553c8ee-6366-4b9c-a32f-42d75543fb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:25.697379 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553c8ee-6366-4b9c-a32f-42d75543fb4f None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:25.697800 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553c8ee-6366-4b9c-a32f-42d75543fb4f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:25.698252 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8553c8ee-6366-4b9c-a32f-42d75543fb4f None admin] PATH_INFO: `/v3/projects/d18a5117a03b4f21ab3290a15680de66/users/b6a1f44f6fe94c09b696867871fd5b47/roles/faf1138f11d34abe8a862f1a1f7c140a` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:25.711894 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ecb303-9dc0-4ae6-9265-0968ec188ae9 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:25.714128 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553c8ee-6366-4b9c-a32f-42d75543fb4f None admin] RBAC: Authorizing `identity:create_grant(project_id=d18a5117a03b4f21ab3290a15680de66, user_id=b6a1f44f6fe94c09b696867871fd5b47, role_id=faf1138f11d34abe8a862f1a1f7c140a)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:25.715630 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8553c8ee-6366-4b9c-a32f-42d75543fb4f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:25.745870 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3aff6a-662a-4d46-ab40-5a9decccb0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:25.746499 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3aff6a-662a-4d46-ab40-5a9decccb0e5 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:25.746815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3aff6a-662a-4d46-ab40-5a9decccb0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:25.747096 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3aff6a-662a-4d46-ab40-5a9decccb0e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:25.753556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3aff6a-662a-4d46-ab40-5a9decccb0e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:25.755011 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3aff6a-662a-4d46-ab40-5a9decccb0e5 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:25.768861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-4a3aff6a-662a-4d46-ab40-5a9decccb0e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:25.768861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:25.768861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:25.768861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:25.768861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:25.768861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:25.768861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:25.768861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:25.768861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:25.768861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:25.768861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:25.768861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:25.768861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:14:25.768861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:14:25.768861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:14:25.769927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:14:25.769927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:25.769927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:25.769927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:25.769927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:25.769927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:25.769927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:25.769927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:25.769927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:25.769927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:25.769927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:25.769927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:25.769927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:25.769927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:25.769927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:25.769927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:25.769927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:25.771286 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:14:25.771286 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:14:25.771286 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:14:25.771286 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:14:25.771286 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:14:25.771286 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:14:25.771286 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:25.771286 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:25.771286 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 297/603] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:25.771286 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3aff6a-662a-4d46-ab40-5a9decccb0e5 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:25.790786 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 307/604] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Sep 2 09:14:25 2022] PUT /identity/v3/projects/d18a5117a03b4f21ab3290a15680de66/users/b6a1f44f6fe94c09b696867871fd5b47/roles/faf1138f11d34abe8a862f1a1f7c140a => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:25.792149 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715f547-701a-451f-aa05-0fcff11a20fb None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:25.792434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715f547-701a-451f-aa05-0fcff11a20fb None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:25.792705 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715f547-701a-451f-aa05-0fcff11a20fb None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:25.794139 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 308/605] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:25.795642 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715f547-701a-451f-aa05-0fcff11a20fb None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:25.825351 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:25.881457 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:25.896777 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3aff6a-662a-4d46-ab40-5a9decccb0e5 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:25.935054 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715f547-701a-451f-aa05-0fcff11a20fb None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:25.949449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d40cf-108e-4787-8cb2-66d77ba2d5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:25.950139 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d40cf-108e-4787-8cb2-66d77ba2d5bf None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:25.950574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d40cf-108e-4787-8cb2-66d77ba2d5bf None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:25.951043 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d40cf-108e-4787-8cb2-66d77ba2d5bf None admin] PATH_INFO: `/v3/projects/d18a5117a03b4f21ab3290a15680de66/users/31b555bb94ce44ce918cbf2a012576a7/roles/faf1138f11d34abe8a862f1a1f7c140a` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:25.972768 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d40cf-108e-4787-8cb2-66d77ba2d5bf None admin] RBAC: Authorizing `identity:create_grant(project_id=d18a5117a03b4f21ab3290a15680de66, user_id=31b555bb94ce44ce918cbf2a012576a7, role_id=faf1138f11d34abe8a862f1a1f7c140a)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:25.974576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80d40cf-108e-4787-8cb2-66d77ba2d5bf None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:25.979445 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a70066-38fc-43a5-93f7-f4904d2024b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:25.980068 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a70066-38fc-43a5-93f7-f4904d2024b6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:25.980360 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a70066-38fc-43a5-93f7-f4904d2024b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:25.980627 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a70066-38fc-43a5-93f7-f4904d2024b6 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:25.981429 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a70066-38fc-43a5-93f7-f4904d2024b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:25.981636 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a70066-38fc-43a5-93f7-f4904d2024b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:25.983096 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a70066-38fc-43a5-93f7-f4904d2024b6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:25.998804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 309/606] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:14:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:25.998804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12119268-e03a-4643-b315-90a523e7f3ed None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:25.998804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12119268-e03a-4643-b315-90a523e7f3ed None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:25.998804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12119268-e03a-4643-b315-90a523e7f3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:26.018501 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-12119268-e03a-4643-b315-90a523e7f3ed None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:26.019875 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-12119268-e03a-4643-b315-90a523e7f3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:26.059646 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 298/607] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Sep 2 09:14:25 2022] PUT /identity/v3/projects/d18a5117a03b4f21ab3290a15680de66/users/31b555bb94ce44ce918cbf2a012576a7/roles/faf1138f11d34abe8a862f1a1f7c140a => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:26.060468 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d40cf-108e-4787-8cb2-66d77ba2d5bf None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:26.105940 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 310/608] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:26.107406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81b6cd-c977-495b-8839-41d5a7e84a85 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:26.107739 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81b6cd-c977-495b-8839-41d5a7e84a85 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:26.108065 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f81b6cd-c977-495b-8839-41d5a7e84a85 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:26.114153 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 311/609] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:26.115827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8604b8-2178-4f99-9f7a-6270dd1cf122 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:26.116108 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8604b8-2178-4f99-9f7a-6270dd1cf122 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:26.116913 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8604b8-2178-4f99-9f7a-6270dd1cf122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:26.137497 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6b8604b8-2178-4f99-9f7a-6270dd1cf122 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:26.138899 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6b8604b8-2178-4f99-9f7a-6270dd1cf122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:26.193138 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:26.216458 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 312/610] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 02 09:14:26.222826 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:26.235318 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d40cf-108e-4787-8cb2-66d77ba2d5bf None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:26.274447 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ff932-88e2-4e38-940c-c2c83a2f9de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:26.275068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ff932-88e2-4e38-940c-c2c83a2f9de3 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:26.275353 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ff932-88e2-4e38-940c-c2c83a2f9de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:26.275630 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ff932-88e2-4e38-940c-c2c83a2f9de3 None admin] PATH_INFO: `/v3/projects/d18a5117a03b4f21ab3290a15680de66/users/dfea2ee0c9e84b48ac9bafe3173d937a/roles/faf1138f11d34abe8a862f1a1f7c140a` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:26.290761 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399ff932-88e2-4e38-940c-c2c83a2f9de3 None admin] RBAC: Authorizing `identity:create_grant(project_id=d18a5117a03b4f21ab3290a15680de66, user_id=dfea2ee0c9e84b48ac9bafe3173d937a, role_id=faf1138f11d34abe8a862f1a1f7c140a)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:26.292206 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399ff932-88e2-4e38-940c-c2c83a2f9de3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:26.367588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 299/611] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Sep 2 09:14:26 2022] PUT /identity/v3/projects/d18a5117a03b4f21ab3290a15680de66/users/dfea2ee0c9e84b48ac9bafe3173d937a/roles/faf1138f11d34abe8a862f1a1f7c140a => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:26.494267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8604b8-2178-4f99-9f7a-6270dd1cf122 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:26.533675 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ff932-88e2-4e38-940c-c2c83a2f9de3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:26.603179 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8604b8-2178-4f99-9f7a-6270dd1cf122 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:26.639148 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88314be2-346d-4527-b3e7-92b0f587d263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:26.639786 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88314be2-346d-4527-b3e7-92b0f587d263 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:26.640063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88314be2-346d-4527-b3e7-92b0f587d263 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:26.640332 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88314be2-346d-4527-b3e7-92b0f587d263 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:26.651690 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88314be2-346d-4527-b3e7-92b0f587d263 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:26.656520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-88314be2-346d-4527-b3e7-92b0f587d263 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:26.656520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:26.656520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:26.656520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:26.656520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:26.656520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:26.656520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:26.656520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:26.656520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:26.656520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:26.656520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:26.656520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:26.656520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:26.656520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:26.656520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:26.656520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:26.657682 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:26.657682 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:26.657682 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:26.657682 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:26.657682 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:26.657682 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:26.657682 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:26.657682 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:26.657682 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:26.657682 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:26.657682 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:26.657682 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:26.657682 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:26.657682 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:26.657682 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:26.657682 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:26.658766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:26.658766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:26.658766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:26.658766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:26.658766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:26.658766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:26.658766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:26.658766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:26.658766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:26.658766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:26.658766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:26.658766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:26.658766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 313/612] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Sep 2 09:14:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:26.663576 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88314be2-346d-4527-b3e7-92b0f587d263 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:26.676163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ff932-88e2-4e38-940c-c2c83a2f9de3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:26.709662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:26.716729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53524b-4827-42e4-9b6b-dd86e4b8e763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:26.717550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53524b-4827-42e4-9b6b-dd86e4b8e763 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:26.718054 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53524b-4827-42e4-9b6b-dd86e4b8e763 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:26.718486 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53524b-4827-42e4-9b6b-dd86e4b8e763 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:26.726898 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:26.730515 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53524b-4827-42e4-9b6b-dd86e4b8e763 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:26.735445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-fa53524b-4827-42e4-9b6b-dd86e4b8e763 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:26.735445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:26.735445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:26.735445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:26.735445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:26.735445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:26.735445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:26.735445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:26.735445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:26.735445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:26.735445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:26.735445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:26.735445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:26.735445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:26.735445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:26.735445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:26.736570 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:26.736570 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:26.736570 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:26.736570 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:26.736570 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:26.736570 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:26.736570 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:26.736570 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:26.736570 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:26.736570 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:26.736570 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:26.736570 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:26.736570 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:26.736570 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:26.736570 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:26.736570 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:26.737606 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:26.737606 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:26.737606 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:26.737606 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:26.737606 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:26.737606 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:26.737606 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:26.737606 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:26.737606 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:26.737606 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:26.737606 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:26.737606 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:26.739784 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 300/613] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Sep 2 09:14:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:26.746047 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53524b-4827-42e4-9b6b-dd86e4b8e763 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:26.769406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88314be2-346d-4527-b3e7-92b0f587d263 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:26.804848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4301a-317a-42c2-96d6-76f195392c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:26.805375 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4301a-317a-42c2-96d6-76f195392c03 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:26.805654 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4301a-317a-42c2-96d6-76f195392c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:26.805980 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4301a-317a-42c2-96d6-76f195392c03 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:26.806642 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e4301a-317a-42c2-96d6-76f195392c03 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:26.806966 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e4301a-317a-42c2-96d6-76f195392c03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:26.808324 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e4301a-317a-42c2-96d6-76f195392c03 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:26.816460 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 314/614] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Sep 2 09:14:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:14:26.822516 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4301a-317a-42c2-96d6-76f195392c03 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:26.910688 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53524b-4827-42e4-9b6b-dd86e4b8e763 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:26.923906 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4301a-317a-42c2-96d6-76f195392c03 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:26.926217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:26.959312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bdfe01-d08d-482a-9eed-4ff71d4fa6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:26.959932 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bdfe01-d08d-482a-9eed-4ff71d4fa6cf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:26.960232 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bdfe01-d08d-482a-9eed-4ff71d4fa6cf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:26.960507 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bdfe01-d08d-482a-9eed-4ff71d4fa6cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:26.971244 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bdfe01-d08d-482a-9eed-4ff71d4fa6cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:26.972893 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bdfe01-d08d-482a-9eed-4ff71d4fa6cf None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:26.977608 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538af728-8eef-48f1-a847-24e7ea0c6e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:26.978275 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538af728-8eef-48f1-a847-24e7ea0c6e63 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:26.978603 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538af728-8eef-48f1-a847-24e7ea0c6e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:26.978908 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538af728-8eef-48f1-a847-24e7ea0c6e63 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:26.979580 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538af728-8eef-48f1-a847-24e7ea0c6e63 None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:26.980030 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538af728-8eef-48f1-a847-24e7ea0c6e63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:26.981528 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538af728-8eef-48f1-a847-24e7ea0c6e63 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:26.987784 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-c6bdfe01-d08d-482a-9eed-4ff71d4fa6cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:26.987784 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:26.987784 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:26.987784 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:26.987784 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:26.987784 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:26.987784 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:26.987784 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:26.987784 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:26.987784 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:26.987784 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:26.987784 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:26.987784 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:26.987784 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:26.987784 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:26.987784 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:26.988757 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:26.988757 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:26.988757 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:26.988757 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:26.988757 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:26.988757 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:26.988757 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:26.988757 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:26.988757 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:26.988757 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:26.988757 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:26.988757 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:26.988757 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:26.988757 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:26.988757 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:26.988757 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:26.988757 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:26.989663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:26.989663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:26.989663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:26.989663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 315/615] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:26.995308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bdfe01-d08d-482a-9eed-4ff71d4fa6cf None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:27.001686 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 301/616] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Sep 2 09:14:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:14:27.008140 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538af728-8eef-48f1-a847-24e7ea0c6e63 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:27.098479 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bdfe01-d08d-482a-9eed-4ff71d4fa6cf None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:27.120094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:27.135712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce92df0-fd46-4b18-ba1d-9907073284a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:27.136327 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce92df0-fd46-4b18-ba1d-9907073284a8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:27.136607 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce92df0-fd46-4b18-ba1d-9907073284a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:27.136878 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce92df0-fd46-4b18-ba1d-9907073284a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:27.137624 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce92df0-fd46-4b18-ba1d-9907073284a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:27.137925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce92df0-fd46-4b18-ba1d-9907073284a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:27.139378 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce92df0-fd46-4b18-ba1d-9907073284a8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:27.151964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 316/617] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:14:27.157320 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce92df0-fd46-4b18-ba1d-9907073284a8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:27.175657 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:27.196184 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538af728-8eef-48f1-a847-24e7ea0c6e63 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:27.261992 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce92df0-fd46-4b18-ba1d-9907073284a8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:27.283975 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c721cf8-ba9d-481e-ac02-c02cc06ba8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:27.285271 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c721cf8-ba9d-481e-ac02-c02cc06ba8db None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:27.285729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c721cf8-ba9d-481e-ac02-c02cc06ba8db None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:27.286186 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c721cf8-ba9d-481e-ac02-c02cc06ba8db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:27.297752 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a8c270-5c62-49fb-98d3-ee201c9cd5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:27.298477 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8c270-5c62-49fb-98d3-ee201c9cd5aa None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:27.298838 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8c270-5c62-49fb-98d3-ee201c9cd5aa None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:27.299178 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a8c270-5c62-49fb-98d3-ee201c9cd5aa None admin] PATH_INFO: `/v3/users/nova` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:27.307294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a8c270-5c62-49fb-98d3-ee201c9cd5aa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:27.308834 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a8c270-5c62-49fb-98d3-ee201c9cd5aa None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:27.315755 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c721cf8-ba9d-481e-ac02-c02cc06ba8db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:27.317435 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c721cf8-ba9d-481e-ac02-c02cc06ba8db None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:27.321063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-28a8c270-5c62-49fb-98d3-ee201c9cd5aa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 02 09:14:27.321063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:27.321063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:27.321063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:27.321063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:27.321063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:27.321063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:27.321063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:27.321063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:27.321063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:27.321063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:27.321063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:27.321063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:14:27.321063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:14:27.321063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:14:27.321063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:14:27.321964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:27.321964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:27.321964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:14:27.321964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:27.321964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:14:27.321964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:27.321964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:27.321964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:27.321964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:27.321964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:27.321964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:27.321964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:27.321964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:27.321964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:27.321964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:27.321964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:27.321964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:27.321964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:27.323441 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:27.323441 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:27.323441 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:14:27.323441 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:14:27.323441 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:14:27.323441 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:14:27.323441 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:14:27.323441 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:14:27.323441 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 02 09:14:27.323441 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:27.323441 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 317/618] 10.222.0.57 () {56 vars in 1093 bytes} [Fri Sep 2 09:14:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:14:27.328341 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a8c270-5c62-49fb-98d3-ee201c9cd5aa None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:27.353526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-8c721cf8-ba9d-481e-ac02-c02cc06ba8db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:27.353526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:27.353526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:27.353526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:27.353526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:27.353526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:27.353526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:27.353526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:27.353526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:27.353526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:27.353526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:27.353526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:27.353526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:27.353526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:27.353526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:27.353526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:27.354496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:27.354496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:27.354496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:27.354496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:27.354496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:27.354496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:27.354496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:27.354496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:27.354496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:27.354496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:27.354496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:27.354496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:27.354496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:27.354496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:27.354496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:27.354496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:27.354496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:27.363654 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:27.363654 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:27.363654 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:27.365589 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 302/619] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:14:27.371790 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c721cf8-ba9d-481e-ac02-c02cc06ba8db None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:27.418147 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:27.456239 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a8c270-5c62-49fb-98d3-ee201c9cd5aa None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:27.490319 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7826e6-b757-4fca-b7af-781f2d1dd21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:27.490960 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7826e6-b757-4fca-b7af-781f2d1dd21b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:27.491261 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7826e6-b757-4fca-b7af-781f2d1dd21b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:27.492118 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7826e6-b757-4fca-b7af-781f2d1dd21b None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:27.492905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7826e6-b757-4fca-b7af-781f2d1dd21b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:27.493217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7826e6-b757-4fca-b7af-781f2d1dd21b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:27.494627 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7826e6-b757-4fca-b7af-781f2d1dd21b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:27.512132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 318/620] 10.222.0.57 () {56 vars in 1133 bytes} [Fri Sep 2 09:14:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:14:27.517534 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7826e6-b757-4fca-b7af-781f2d1dd21b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:27.565359 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c721cf8-ba9d-481e-ac02-c02cc06ba8db None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:27.617779 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:27.618193 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7826e6-b757-4fca-b7af-781f2d1dd21b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:27.640554 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48609f-506d-4d58-b1d1-ec7ee67c195b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:27.641177 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48609f-506d-4d58-b1d1-ec7ee67c195b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:27.641461 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48609f-506d-4d58-b1d1-ec7ee67c195b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:27.641738 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48609f-506d-4d58-b1d1-ec7ee67c195b None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:27.642396 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c48609f-506d-4d58-b1d1-ec7ee67c195b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:27.642721 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c48609f-506d-4d58-b1d1-ec7ee67c195b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:27.644151 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c48609f-506d-4d58-b1d1-ec7ee67c195b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:27.654361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c76014-2c30-4e78-93e7-29de314810c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:27.655024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c76014-2c30-4e78-93e7-29de314810c6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:27.655303 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c76014-2c30-4e78-93e7-29de314810c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:27.655569 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c76014-2c30-4e78-93e7-29de314810c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:27.671063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:27.673297 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 303/621] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:14:27.678636 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48609f-506d-4d58-b1d1-ec7ee67c195b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:27.784284 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c76014-2c30-4e78-93e7-29de314810c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:27.785740 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c76014-2c30-4e78-93e7-29de314810c6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:27.868674 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48609f-506d-4d58-b1d1-ec7ee67c195b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:27.937360 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28928efc-e9ae-4e17-b9a0-e0bb3add3901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:27.937998 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28928efc-e9ae-4e17-b9a0-e0bb3add3901 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:27.938275 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28928efc-e9ae-4e17-b9a0-e0bb3add3901 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:27.940099 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28928efc-e9ae-4e17-b9a0-e0bb3add3901 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:27.958576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28928efc-e9ae-4e17-b9a0-e0bb3add3901 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:27.960731 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28928efc-e9ae-4e17-b9a0-e0bb3add3901 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:27.987237 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 319/622] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:14:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 02 09:14:27.988637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-28928efc-e9ae-4e17-b9a0-e0bb3add3901 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 02 09:14:27.988637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:27.988637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:27.988637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:27.988637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:27.988637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:27.988637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:27.988637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:27.988637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:27.988637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:27.988637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:27.988637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:27.988637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:14:27.988637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:14:27.988637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:14:27.988637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:14:27.989579 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:27.989579 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:27.989579 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:14:27.989579 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:27.989579 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:14:27.989579 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:27.989579 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:27.989579 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:27.989579 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:27.989579 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:27.989579 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:27.989579 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:27.989579 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:27.989579 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:27.989579 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:27.989579 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:27.989579 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:27.989579 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:27.990524 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:27.990524 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:27.990524 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:14:27.990524 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:14:27.990524 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:14:27.990524 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:14:27.990524 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:14:27.990524 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:14:27.990524 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 02 09:14:27.990524 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:27.990524 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 304/623] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:14:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:14:27.991891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c76014-2c30-4e78-93e7-29de314810c6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:27.995754 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28928efc-e9ae-4e17-b9a0-e0bb3add3901 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:28.035972 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:28.055848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:28.133032 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28928efc-e9ae-4e17-b9a0-e0bb3add3901 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:28.146328 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c76014-2c30-4e78-93e7-29de314810c6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:28.168982 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17951f5d-14c6-448f-acde-ab35c73ce696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:28.169593 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17951f5d-14c6-448f-acde-ab35c73ce696 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:28.169875 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17951f5d-14c6-448f-acde-ab35c73ce696 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:28.170152 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17951f5d-14c6-448f-acde-ab35c73ce696 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:28.170921 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17951f5d-14c6-448f-acde-ab35c73ce696 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:28.171232 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17951f5d-14c6-448f-acde-ab35c73ce696 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:28.172649 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17951f5d-14c6-448f-acde-ab35c73ce696 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:28.184083 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411e1a0-eb14-4b5e-a130-c5299dff70c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:28.184727 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411e1a0-eb14-4b5e-a130-c5299dff70c3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:28.185003 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411e1a0-eb14-4b5e-a130-c5299dff70c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:28.185274 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411e1a0-eb14-4b5e-a130-c5299dff70c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:28.190513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 305/624] 10.222.0.57 () {56 vars in 1135 bytes} [Fri Sep 2 09:14:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:14:28.191869 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a40d2c-3d55-4ad5-9e99-b925fadc3a3e None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:28.192162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a40d2c-3d55-4ad5-9e99-b925fadc3a3e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:28.192439 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a40d2c-3d55-4ad5-9e99-b925fadc3a3e None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:28.193933 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 306/625] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:28.196014 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a40d2c-3d55-4ad5-9e99-b925fadc3a3e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:28.201182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411e1a0-eb14-4b5e-a130-c5299dff70c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:28.202659 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411e1a0-eb14-4b5e-a130-c5299dff70c3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:28.219368 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2411e1a0-eb14-4b5e-a130-c5299dff70c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:28.219368 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:28.219368 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:28.219368 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:28.219368 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:28.219368 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:28.219368 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:28.219368 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:28.219368 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:28.219368 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:28.219368 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:28.219368 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:28.219368 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:28.219368 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:28.219368 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:28.219368 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:28.220871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:28.220871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:28.220871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:28.220871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:28.220871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:28.220871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:28.220871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:28.220871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:28.220871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:28.220871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:28.220871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:28.220871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:28.220871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:28.220871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:28.220871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:28.220871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:28.220871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:28.223127 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:28.223127 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:28.223127 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:28.223127 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 320/626] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:28.223127 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a20aa-db1b-4867-a3ef-669b3b960adc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:28.223127 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a20aa-db1b-4867-a3ef-669b3b960adc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:28.223127 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751a20aa-db1b-4867-a3ef-669b3b960adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:28.242343 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-751a20aa-db1b-4867-a3ef-669b3b960adc None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:28.243629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-751a20aa-db1b-4867-a3ef-669b3b960adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:28.294167 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:28.304413 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a40d2c-3d55-4ad5-9e99-b925fadc3a3e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:28.326954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 321/627] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:28.328086 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a20aa-db1b-4867-a3ef-669b3b960adc None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:28.342730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d7d32-2344-48d5-b03d-b61755df5037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:28.342730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d7d32-2344-48d5-b03d-b61755df5037 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:28.342730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d7d32-2344-48d5-b03d-b61755df5037 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:28.342730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444d7d32-2344-48d5-b03d-b61755df5037 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:28.351891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:28.358728 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d7d32-2344-48d5-b03d-b61755df5037 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:28.358728 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444d7d32-2344-48d5-b03d-b61755df5037 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:28.374729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-444d7d32-2344-48d5-b03d-b61755df5037 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:28.374729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:28.374729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:28.374729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:28.374729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:28.374729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:28.374729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:28.374729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:28.374729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:28.374729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:28.374729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:28.374729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:28.374729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:28.374729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:28.374729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:28.374729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:28.375718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:28.375718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:28.375718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:28.375718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:28.375718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:28.375718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:28.375718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:28.375718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:28.375718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:28.375718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:28.375718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:28.375718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:28.375718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:28.375718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:28.375718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:28.375718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:28.375718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:28.376681 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:28.376681 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:28.376681 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:28.376681 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 307/628] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:28.379241 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d7d32-2344-48d5-b03d-b61755df5037 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:28.482629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751a20aa-db1b-4867-a3ef-669b3b960adc None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:28.519053 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a77745-3dfb-40cf-9396-1860a1f350fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:28.519473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444d7d32-2344-48d5-b03d-b61755df5037 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:28.519700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a77745-3dfb-40cf-9396-1860a1f350fd None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:28.519986 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a77745-3dfb-40cf-9396-1860a1f350fd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:28.520263 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a77745-3dfb-40cf-9396-1860a1f350fd None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:28.521016 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a77745-3dfb-40cf-9396-1860a1f350fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:28.521327 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a77745-3dfb-40cf-9396-1860a1f350fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:28.523391 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a77745-3dfb-40cf-9396-1860a1f350fd None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:28.535807 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 322/629] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:28.537198 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e113ff8-ae2d-45d3-b292-df73223c88c5 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:28.537618 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e113ff8-ae2d-45d3-b292-df73223c88c5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:28.537895 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e113ff8-ae2d-45d3-b292-df73223c88c5 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:28.543904 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 323/630] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:28.544794 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e113ff8-ae2d-45d3-b292-df73223c88c5 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:28.555660 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3217d-7f0c-429e-95dc-b21521463de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:28.556274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3217d-7f0c-429e-95dc-b21521463de2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:28.560739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3217d-7f0c-429e-95dc-b21521463de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:28.560739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3217d-7f0c-429e-95dc-b21521463de2 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:28.560739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc3217d-7f0c-429e-95dc-b21521463de2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:28.560739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc3217d-7f0c-429e-95dc-b21521463de2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:28.560739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc3217d-7f0c-429e-95dc-b21521463de2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:28.572203 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 308/631] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:14:28.573800 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523db55-059d-4f3b-9a1a-bfa19931f6e5 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:28.574088 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523db55-059d-4f3b-9a1a-bfa19931f6e5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:28.574367 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0523db55-059d-4f3b-9a1a-bfa19931f6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:28.581609 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:28.593242 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0523db55-059d-4f3b-9a1a-bfa19931f6e5 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:28.595150 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0523db55-059d-4f3b-9a1a-bfa19931f6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:28.633987 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:28.660100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e113ff8-ae2d-45d3-b292-df73223c88c5 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:28.670408 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 309/632] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:28.670408 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0523db55-059d-4f3b-9a1a-bfa19931f6e5 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:28.719804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e29bb-03d5-443c-969b-104a747d7d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:28.720445 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e29bb-03d5-443c-969b-104a747d7d78 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:28.720726 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e29bb-03d5-443c-969b-104a747d7d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:28.720997 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e29bb-03d5-443c-969b-104a747d7d78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:28.728457 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e29bb-03d5-443c-969b-104a747d7d78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:28.729893 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e29bb-03d5-443c-969b-104a747d7d78 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:28.741976 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-106e29bb-03d5-443c-969b-104a747d7d78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:28.741976 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:28.741976 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:28.741976 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:28.741976 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:28.741976 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:28.741976 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:28.741976 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:28.741976 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:28.741976 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:28.741976 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:28.741976 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:28.741976 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:14:28.741976 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:14:28.741976 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:14:28.742901 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:14:28.742901 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:28.742901 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:28.742901 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:28.742901 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:28.742901 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:28.742901 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:28.742901 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:28.742901 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:28.742901 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:28.742901 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:28.742901 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:28.742901 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:28.742901 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:28.742901 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:28.742901 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:28.742901 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:28.743796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:14:28.743796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:14:28.743796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:14:28.743796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:14:28.743796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:14:28.743796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:14:28.743796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:28.743796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:28.743796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 324/633] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:28.744318 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e29bb-03d5-443c-969b-104a747d7d78 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:28.770983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0523db55-059d-4f3b-9a1a-bfa19931f6e5 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:28.809682 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0416810f-5f25-41b6-a72f-45c61bb3fbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:28.810320 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416810f-5f25-41b6-a72f-45c61bb3fbe1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:28.810697 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416810f-5f25-41b6-a72f-45c61bb3fbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:28.811014 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416810f-5f25-41b6-a72f-45c61bb3fbe1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:28.818118 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0416810f-5f25-41b6-a72f-45c61bb3fbe1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:28.819637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0416810f-5f25-41b6-a72f-45c61bb3fbe1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:28.832024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-0416810f-5f25-41b6-a72f-45c61bb3fbe1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:28.832024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:28.832024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:28.832024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:28.832024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:28.832024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:28.832024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:28.832024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:28.832024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:28.832024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:28.832024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:28.832024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:28.832024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:14:28.832024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:14:28.832024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:14:28.832926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:14:28.832926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:28.832926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:28.832926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:28.832926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:28.832926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:28.832926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:28.832926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:28.832926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:28.832926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:28.832926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:28.832926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:28.832926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:28.832926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:28.832926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:28.832926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:28.832926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:28.833829 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:14:28.833829 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:14:28.833829 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:14:28.833829 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:14:28.833829 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:14:28.833829 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:14:28.833829 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:28.833829 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:28.833829 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 310/634] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:28.834344 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0416810f-5f25-41b6-a72f-45c61bb3fbe1 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:28.835602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:28.849762 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e29bb-03d5-443c-969b-104a747d7d78 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:28.884777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93d97f-2f1a-4092-b952-9503be1e2ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:28.885405 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93d97f-2f1a-4092-b952-9503be1e2ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:28.885683 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93d97f-2f1a-4092-b952-9503be1e2ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:28.885951 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93d97f-2f1a-4092-b952-9503be1e2ff0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:28.888787 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:28.896868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93d97f-2f1a-4092-b952-9503be1e2ff0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:28.903622 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-cb93d97f-2f1a-4092-b952-9503be1e2ff0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:28.903622 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:28.903622 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:28.903622 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:28.903622 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:28.903622 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:28.903622 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:28.903622 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:28.903622 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:28.903622 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:28.903622 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:28.903622 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:28.903622 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:28.903622 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:28.903622 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:28.903622 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:28.904506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:28.904506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:28.904506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:28.904506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:28.904506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:28.904506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:28.904506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:28.904506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:28.904506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:28.904506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:28.904506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:28.904506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:28.904506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:28.904506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:28.904506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:28.904506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:28.905517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:28.905517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:28.905517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:28.905517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:28.905517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:28.905517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:28.905517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:28.905517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:28.905517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:28.905517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:28.905517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:28.905517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:28.905517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 325/635] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Sep 2 09:14:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:28.905517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93d97f-2f1a-4092-b952-9503be1e2ff0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:28.954983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0416810f-5f25-41b6-a72f-45c61bb3fbe1 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:28.990680 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10effe07-5a86-4b39-b7c9-300c7d645c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:28.991868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10effe07-5a86-4b39-b7c9-300c7d645c1f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:28.992204 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10effe07-5a86-4b39-b7c9-300c7d645c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:28.992424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10effe07-5a86-4b39-b7c9-300c7d645c1f None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:28.993108 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10effe07-5a86-4b39-b7c9-300c7d645c1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:28.993410 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10effe07-5a86-4b39-b7c9-300c7d645c1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:28.994849 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10effe07-5a86-4b39-b7c9-300c7d645c1f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:29.005730 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93d97f-2f1a-4092-b952-9503be1e2ff0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:29.007074 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 311/636] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:14:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:29.008253 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069a810-c8d2-4ead-98a7-59c6c859bad2 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:29.008541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069a810-c8d2-4ead-98a7-59c6c859bad2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:29.008816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069a810-c8d2-4ead-98a7-59c6c859bad2 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:29.010208 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 312/637] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:29.010728 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069a810-c8d2-4ead-98a7-59c6c859bad2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:29.043821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f86b9-bd43-43d8-8a7b-2ea02966e3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:29.044461 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f86b9-bd43-43d8-8a7b-2ea02966e3ef None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:29.044748 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f86b9-bd43-43d8-8a7b-2ea02966e3ef None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:29.045023 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f86b9-bd43-43d8-8a7b-2ea02966e3ef None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:29.045716 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f86b9-bd43-43d8-8a7b-2ea02966e3ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:29.046028 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f86b9-bd43-43d8-8a7b-2ea02966e3ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:29.047500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f86b9-bd43-43d8-8a7b-2ea02966e3ef None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:29.059585 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 326/638] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:14:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:29.060381 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f86b9-bd43-43d8-8a7b-2ea02966e3ef None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:29.067442 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:29.113512 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069a810-c8d2-4ead-98a7-59c6c859bad2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:29.117268 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:29.147108 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d411b-7d0f-4501-bb29-081efb6a347c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:29.148302 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d411b-7d0f-4501-bb29-081efb6a347c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:29.148584 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d411b-7d0f-4501-bb29-081efb6a347c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:29.148880 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d411b-7d0f-4501-bb29-081efb6a347c None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:29.149538 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d411b-7d0f-4501-bb29-081efb6a347c None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:29.149840 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d411b-7d0f-4501-bb29-081efb6a347c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:29.151285 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d411b-7d0f-4501-bb29-081efb6a347c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:29.159116 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 313/639] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Sep 2 09:14:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:29.159339 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f86b9-bd43-43d8-8a7b-2ea02966e3ef None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:29.160017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d411b-7d0f-4501-bb29-081efb6a347c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:29.194617 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c64f563-4993-4301-a1ca-e6e2a23ecf5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:29.195281 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64f563-4993-4301-a1ca-e6e2a23ecf5d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:29.195563 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64f563-4993-4301-a1ca-e6e2a23ecf5d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:29.195839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c64f563-4993-4301-a1ca-e6e2a23ecf5d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:29.206050 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c64f563-4993-4301-a1ca-e6e2a23ecf5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:29.211352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4c64f563-4993-4301-a1ca-e6e2a23ecf5d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:29.211352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:29.211352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:29.211352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:29.211352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:29.211352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:29.211352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:29.211352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:29.211352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:29.211352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:29.211352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:29.211352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:29.211352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:29.211352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:29.211352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:29.211352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:29.212267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:29.212267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:29.212267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:29.212267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:29.212267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:29.212267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:29.212267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:29.212267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:29.212267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:29.212267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:29.212267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:29.212267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:29.212267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:29.212267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:29.212267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:29.212267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:29.213079 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:29.213079 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:29.213079 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:29.213079 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:29.213079 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:29.213079 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:29.213079 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:29.213079 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:29.213079 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:29.213079 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:29.213079 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:29.213079 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:29.213079 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 327/640] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Sep 2 09:14:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:29.214661 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f87a28-9d16-4486-b759-3e215a33ec2f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:29.214661 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f87a28-9d16-4486-b759-3e215a33ec2f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:29.214661 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f87a28-9d16-4486-b759-3e215a33ec2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:29.231717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-52f87a28-9d16-4486-b759-3e215a33ec2f None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:29.232916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-52f87a28-9d16-4486-b759-3e215a33ec2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:29.258763 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d411b-7d0f-4501-bb29-081efb6a347c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:29.274863 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:29.292410 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e517a-b71f-41bd-9492-2eae438e326a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:29.293017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e517a-b71f-41bd-9492-2eae438e326a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:29.293300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e517a-b71f-41bd-9492-2eae438e326a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:29.293577 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e517a-b71f-41bd-9492-2eae438e326a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:29.294298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e517a-b71f-41bd-9492-2eae438e326a None admin] RBAC: Adding query filter params (role.id=faf1138f11d34abe8a862f1a1f7c140a, scope.project.id=d18a5117a03b4f21ab3290a15680de66, user.id=1da355c8be2d496bb08feb0fe74e04b9) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:29.294867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e517a-b71f-41bd-9492-2eae438e326a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:29.296055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e517a-b71f-41bd-9492-2eae438e326a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:29.299915 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 328/641] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:29.300721 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f87a28-9d16-4486-b759-3e215a33ec2f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:29.311080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 314/642] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:14:29 2022] GET /identity/v3/role_assignments?user.id=1da355c8be2d496bb08feb0fe74e04b9&scope.project.id=d18a5117a03b4f21ab3290a15680de66&role.id=faf1138f11d34abe8a862f1a1f7c140a => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:29.311080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e517a-b71f-41bd-9492-2eae438e326a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:29.354741 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:29.414843 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f87a28-9d16-4486-b759-3e215a33ec2f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:29.424613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e517a-b71f-41bd-9492-2eae438e326a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:29.453685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26fd615-f2ac-46cf-a524-1bc2d8d7ed80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:29.454845 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fd615-f2ac-46cf-a524-1bc2d8d7ed80 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:29.455455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fd615-f2ac-46cf-a524-1bc2d8d7ed80 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:29.455948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26fd615-f2ac-46cf-a524-1bc2d8d7ed80 None admin] PATH_INFO: `/v3/projects/d18a5117a03b4f21ab3290a15680de66/users/30fe233a7d4c4fbea6ede233bd9ff8c8/roles/faf1138f11d34abe8a862f1a1f7c140a` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:29.463793 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e667d1-5ee1-4fdb-89c5-0a316d48d862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:29.464426 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e667d1-5ee1-4fdb-89c5-0a316d48d862 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:29.464707 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e667d1-5ee1-4fdb-89c5-0a316d48d862 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:29.464983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e667d1-5ee1-4fdb-89c5-0a316d48d862 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:29.473207 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26fd615-f2ac-46cf-a524-1bc2d8d7ed80 None admin] RBAC: Authorizing `identity:create_grant(project_id=d18a5117a03b4f21ab3290a15680de66, user_id=30fe233a7d4c4fbea6ede233bd9ff8c8, role_id=faf1138f11d34abe8a862f1a1f7c140a)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:29.474968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26fd615-f2ac-46cf-a524-1bc2d8d7ed80 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:29.477939 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e667d1-5ee1-4fdb-89c5-0a316d48d862 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:29.479559 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e667d1-5ee1-4fdb-89c5-0a316d48d862 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:29.494407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-50e667d1-5ee1-4fdb-89c5-0a316d48d862 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:29.494407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:29.494407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:29.494407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:29.494407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:29.494407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:29.494407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:29.494407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:29.494407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:29.494407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:29.494407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:29.494407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:29.494407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:29.494407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:29.494407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:29.494407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:29.495509 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:29.495509 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:29.495509 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:29.495509 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:29.495509 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:29.495509 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:29.495509 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:29.495509 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:29.495509 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:29.495509 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:29.495509 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:29.495509 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:29.495509 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:29.495509 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:29.495509 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:29.495509 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:29.495509 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:29.496430 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:29.496430 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:29.496430 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:29.496430 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 315/643] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:29.496430 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e667d1-5ee1-4fdb-89c5-0a316d48d862 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:29.531805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:29.545037 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 329/644] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Sep 2 09:14:29 2022] PUT /identity/v3/projects/d18a5117a03b4f21ab3290a15680de66/users/30fe233a7d4c4fbea6ede233bd9ff8c8/roles/faf1138f11d34abe8a862f1a1f7c140a => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:29.546220 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26fd615-f2ac-46cf-a524-1bc2d8d7ed80 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:29.626294 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e667d1-5ee1-4fdb-89c5-0a316d48d862 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:29.669333 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:29.675512 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26fd615-f2ac-46cf-a524-1bc2d8d7ed80 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:29.679322 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139557c3-2718-4b9f-86fb-c742ea9986f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:29.679978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139557c3-2718-4b9f-86fb-c742ea9986f3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:29.680260 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139557c3-2718-4b9f-86fb-c742ea9986f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:29.680538 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139557c3-2718-4b9f-86fb-c742ea9986f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:29.681227 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139557c3-2718-4b9f-86fb-c742ea9986f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:29.681533 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139557c3-2718-4b9f-86fb-c742ea9986f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:29.682958 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139557c3-2718-4b9f-86fb-c742ea9986f3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:29.695716 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 316/645] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Sep 2 09:14:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:29.696726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139557c3-2718-4b9f-86fb-c742ea9986f3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:29.727619 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ec778-bbd9-4bb4-8b5e-cdb26eae594a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:29.728241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ec778-bbd9-4bb4-8b5e-cdb26eae594a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:29.728524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ec778-bbd9-4bb4-8b5e-cdb26eae594a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:29.728797 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633ec778-bbd9-4bb4-8b5e-cdb26eae594a None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:29.729450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ec778-bbd9-4bb4-8b5e-cdb26eae594a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:29.729754 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ec778-bbd9-4bb4-8b5e-cdb26eae594a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:29.731193 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633ec778-bbd9-4bb4-8b5e-cdb26eae594a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:29.745753 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 330/646] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:29.749248 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ec778-bbd9-4bb4-8b5e-cdb26eae594a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:29.829048 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139557c3-2718-4b9f-86fb-c742ea9986f3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:29.843571 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:29.875002 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633ec778-bbd9-4bb4-8b5e-cdb26eae594a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:29.881255 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b91604-5496-4e94-bf6a-59b7c57f9e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:29.882084 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b91604-5496-4e94-bf6a-59b7c57f9e9c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:29.882442 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b91604-5496-4e94-bf6a-59b7c57f9e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:29.882849 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b91604-5496-4e94-bf6a-59b7c57f9e9c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:29.894272 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b91604-5496-4e94-bf6a-59b7c57f9e9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:29.900640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-70b91604-5496-4e94-bf6a-59b7c57f9e9c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:29.900640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:29.900640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:29.900640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:29.900640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:29.900640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:29.900640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:29.900640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:29.900640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:29.900640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:29.900640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:29.900640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:29.900640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:29.900640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:29.900640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:29.900640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:29.901775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:29.901775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:29.901775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:29.901775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:29.901775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:29.901775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:29.901775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:29.901775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:29.901775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:29.901775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:29.901775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:29.901775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:29.901775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:29.901775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:29.901775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:29.901775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:29.903018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:29.903018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:29.903018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:29.903018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:29.903018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:29.903018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:29.903018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:29.903018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:29.903018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:29.903018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:29.903018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:29.903018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:29.904563 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 317/647] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Sep 2 09:14:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:29.905549 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b91604-5496-4e94-bf6a-59b7c57f9e9c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:29.921425 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d346bf7-bf74-4a7d-bbd5-1d8692fdd9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:29.922109 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d346bf7-bf74-4a7d-bbd5-1d8692fdd9c2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:29.922391 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d346bf7-bf74-4a7d-bbd5-1d8692fdd9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:29.922660 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d346bf7-bf74-4a7d-bbd5-1d8692fdd9c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:29.935524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d346bf7-bf74-4a7d-bbd5-1d8692fdd9c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:29.936973 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d346bf7-bf74-4a7d-bbd5-1d8692fdd9c2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:29.956406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-9d346bf7-bf74-4a7d-bbd5-1d8692fdd9c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:29.956406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:29.956406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:29.956406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:29.956406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:29.956406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:29.956406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:29.956406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:29.956406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:29.956406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:29.956406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:29.956406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:29.956406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:29.956406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:29.956406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:29.956406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:29.957388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:29.957388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:29.957388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:29.957388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:29.957388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:29.957388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:29.957388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:29.957388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:29.957388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:29.957388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:29.957388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:29.957388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:29.957388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:29.957388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:29.957388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:29.957388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:29.957388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:29.958252 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:29.958252 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:29.958252 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:29.962788 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 331/648] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:29.965637 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d346bf7-bf74-4a7d-bbd5-1d8692fdd9c2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:29.979986 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:30.034646 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b91604-5496-4e94-bf6a-59b7c57f9e9c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:30.078698 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4992c4c-eb5d-4fee-a521-1f8fa76d1ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:30.079437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4992c4c-eb5d-4fee-a521-1f8fa76d1ece None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:30.079673 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4992c4c-eb5d-4fee-a521-1f8fa76d1ece None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:30.079937 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4992c4c-eb5d-4fee-a521-1f8fa76d1ece None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:30.089442 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4992c4c-eb5d-4fee-a521-1f8fa76d1ece None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:30.090928 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4992c4c-eb5d-4fee-a521-1f8fa76d1ece None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:30.095415 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d346bf7-bf74-4a7d-bbd5-1d8692fdd9c2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:30.102994 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-a4992c4c-eb5d-4fee-a521-1f8fa76d1ece None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 02 09:14:30.102994 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:30.102994 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:30.102994 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:30.102994 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:30.102994 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:30.102994 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:30.102994 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:30.102994 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:30.102994 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:30.102994 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:30.102994 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:30.102994 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:14:30.102994 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:14:30.102994 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:14:30.102994 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:14:30.103936 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:30.103936 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:30.103936 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:14:30.103936 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:30.103936 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:14:30.103936 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:30.103936 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:30.103936 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:30.103936 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:30.103936 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:30.103936 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:30.103936 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:30.103936 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:30.103936 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:30.103936 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:30.103936 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:30.103936 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:30.103936 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:30.104913 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:30.104913 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:30.104913 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:14:30.104913 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:14:30.104913 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:14:30.104913 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:14:30.104913 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:14:30.104913 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:14:30.104913 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 02 09:14:30.104913 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:30.108939 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 318/649] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:14:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:30.109619 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4992c4c-eb5d-4fee-a521-1f8fa76d1ece None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:30.141636 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2275626-9aec-4464-bf34-ae242b293c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:30.142253 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2275626-9aec-4464-bf34-ae242b293c1d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:30.142536 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2275626-9aec-4464-bf34-ae242b293c1d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:30.142857 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2275626-9aec-4464-bf34-ae242b293c1d None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:30.143529 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2275626-9aec-4464-bf34-ae242b293c1d None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:30.143832 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2275626-9aec-4464-bf34-ae242b293c1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:30.145255 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2275626-9aec-4464-bf34-ae242b293c1d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:30.151092 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:30.159743 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 332/650] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Sep 2 09:14:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:30.160641 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2275626-9aec-4464-bf34-ae242b293c1d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:30.233943 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4992c4c-eb5d-4fee-a521-1f8fa76d1ece None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:30.244190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:30.271636 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427efe9d-dc05-4e54-80c4-2d9fb94dfb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:30.272258 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427efe9d-dc05-4e54-80c4-2d9fb94dfb13 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:30.273078 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427efe9d-dc05-4e54-80c4-2d9fb94dfb13 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:30.273355 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427efe9d-dc05-4e54-80c4-2d9fb94dfb13 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:30.274091 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427efe9d-dc05-4e54-80c4-2d9fb94dfb13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:30.274366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427efe9d-dc05-4e54-80c4-2d9fb94dfb13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:30.275871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427efe9d-dc05-4e54-80c4-2d9fb94dfb13 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:30.286167 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2275626-9aec-4464-bf34-ae242b293c1d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:30.292165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 319/651] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:30.292165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427efe9d-dc05-4e54-80c4-2d9fb94dfb13 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:30.340398 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed63b75-be2f-464a-b967-24c964bfdf64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:30.341764 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed63b75-be2f-464a-b967-24c964bfdf64 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:30.342536 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed63b75-be2f-464a-b967-24c964bfdf64 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:30.343038 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed63b75-be2f-464a-b967-24c964bfdf64 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:30.344258 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed63b75-be2f-464a-b967-24c964bfdf64 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:30.345217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed63b75-be2f-464a-b967-24c964bfdf64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:30.347406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed63b75-be2f-464a-b967-24c964bfdf64 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:30.371777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 333/652] 10.222.0.57 () {56 vars in 1137 bytes} [Fri Sep 2 09:14:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:30.372881 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed63b75-be2f-464a-b967-24c964bfdf64 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:30.379342 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:30.405950 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427efe9d-dc05-4e54-80c4-2d9fb94dfb13 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:30.453117 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6966e-ade7-4fb4-83e9-48260dbc09e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:30.453757 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6966e-ade7-4fb4-83e9-48260dbc09e2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:30.454036 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6966e-ade7-4fb4-83e9-48260dbc09e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:30.454314 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6966e-ade7-4fb4-83e9-48260dbc09e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:30.466223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b6966e-ade7-4fb4-83e9-48260dbc09e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:30.467733 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b6966e-ade7-4fb4-83e9-48260dbc09e2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:30.483123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-61b6966e-ade7-4fb4-83e9-48260dbc09e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:30.483123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:30.483123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:30.483123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:30.483123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:30.483123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:30.483123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:30.483123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:30.483123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:30.483123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:30.483123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:30.483123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:30.483123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:30.483123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:30.483123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:30.483123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:30.484106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:30.484106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:30.484106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:30.484106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:30.484106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:30.484106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:30.484106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:30.484106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:30.484106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:30.484106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:30.484106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:30.484106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:30.484106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:30.484106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:30.484106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:30.484106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:30.484106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:30.484990 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:30.484990 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:30.484990 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:30.489248 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 320/653] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:30.490111 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6966e-ade7-4fb4-83e9-48260dbc09e2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:30.505291 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed63b75-be2f-464a-b967-24c964bfdf64 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:30.546586 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:30.558518 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1dddc2-d484-487f-bc9b-58ad68feffa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:30.562956 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1dddc2-d484-487f-bc9b-58ad68feffa1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:30.562956 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1dddc2-d484-487f-bc9b-58ad68feffa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:30.562956 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1dddc2-d484-487f-bc9b-58ad68feffa1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:30.577921 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1dddc2-d484-487f-bc9b-58ad68feffa1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:30.580708 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1dddc2-d484-487f-bc9b-58ad68feffa1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:30.600934 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ba1dddc2-d484-487f-bc9b-58ad68feffa1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 02 09:14:30.600934 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:30.600934 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:30.600934 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:30.600934 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:30.600934 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:30.600934 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:30.600934 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:30.600934 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:30.600934 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:30.600934 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:30.600934 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:30.600934 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:14:30.600934 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:14:30.600934 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:14:30.600934 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:14:30.602119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:30.602119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:30.602119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:14:30.602119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:30.602119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:14:30.602119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:30.602119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:30.602119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:30.602119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:30.602119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:30.602119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:30.602119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:30.602119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:30.602119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:30.602119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:30.602119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:30.602119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:30.602119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:30.603318 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:30.603318 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:30.603318 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:14:30.603318 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:14:30.603318 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:14:30.603318 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:14:30.603318 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:14:30.603318 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:14:30.603318 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 02 09:14:30.603318 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:30.604986 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 334/654] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:14:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:30.605773 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1dddc2-d484-487f-bc9b-58ad68feffa1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:30.622803 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6966e-ade7-4fb4-83e9-48260dbc09e2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:30.650130 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:30.659728 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123772a-0c9c-4d27-8fd7-0e570ee933da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:30.660521 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123772a-0c9c-4d27-8fd7-0e570ee933da None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:30.663240 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123772a-0c9c-4d27-8fd7-0e570ee933da None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:30.663240 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123772a-0c9c-4d27-8fd7-0e570ee933da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:30.714279 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1dddc2-d484-487f-bc9b-58ad68feffa1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:30.735388 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5123772a-0c9c-4d27-8fd7-0e570ee933da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:30.737134 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5123772a-0c9c-4d27-8fd7-0e570ee933da None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:30.751680 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbba144-aeb1-40c2-a676-4b2a72472eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:30.752529 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbba144-aeb1-40c2-a676-4b2a72472eae None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:30.752959 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbba144-aeb1-40c2-a676-4b2a72472eae None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:30.753625 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbba144-aeb1-40c2-a676-4b2a72472eae None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:30.754094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbba144-aeb1-40c2-a676-4b2a72472eae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:30.754543 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbba144-aeb1-40c2-a676-4b2a72472eae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:30.756025 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbba144-aeb1-40c2-a676-4b2a72472eae None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:30.770180 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 335/655] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:30.771420 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbba144-aeb1-40c2-a676-4b2a72472eae None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:30.822181 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:30.847201 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 321/656] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:14:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:14:30.848278 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123772a-0c9c-4d27-8fd7-0e570ee933da None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:30.876843 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbba144-aeb1-40c2-a676-4b2a72472eae None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:30.881084 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:30.914132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2eadf-dee2-412f-a42b-3992a36a3d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:30.914870 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2eadf-dee2-412f-a42b-3992a36a3d70 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:30.915255 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2eadf-dee2-412f-a42b-3992a36a3d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:30.915631 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2eadf-dee2-412f-a42b-3992a36a3d70 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:30.916480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2eadf-dee2-412f-a42b-3992a36a3d70 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:30.916895 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2eadf-dee2-412f-a42b-3992a36a3d70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:30.918418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc2eadf-dee2-412f-a42b-3992a36a3d70 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:30.940824 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 336/657] 10.222.0.57 () {56 vars in 1137 bytes} [Fri Sep 2 09:14:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:30.940824 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2eadf-dee2-412f-a42b-3992a36a3d70 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:30.966429 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123772a-0c9c-4d27-8fd7-0e570ee933da None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:31.002562 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e38bc2-202f-41b0-9a30-ab6da6faa703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:31.003512 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e38bc2-202f-41b0-9a30-ab6da6faa703 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:31.003940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e38bc2-202f-41b0-9a30-ab6da6faa703 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:31.004381 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e38bc2-202f-41b0-9a30-ab6da6faa703 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:31.012614 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e38bc2-202f-41b0-9a30-ab6da6faa703 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:31.014225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e38bc2-202f-41b0-9a30-ab6da6faa703 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:31.026857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d8e38bc2-202f-41b0-9a30-ab6da6faa703 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 02 09:14:31.026857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:31.026857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:31.026857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:31.026857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:31.026857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:31.026857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:31.026857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:31.026857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:31.026857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:31.026857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:31.026857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:31.026857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:14:31.026857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:14:31.026857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:14:31.026857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:14:31.027878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:31.027878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:31.027878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:14:31.027878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:31.027878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:14:31.027878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:31.027878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:31.027878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:31.027878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:31.027878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:31.027878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:31.027878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:31.027878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:31.027878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:31.027878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:31.027878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:31.027878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:31.027878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:31.028820 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:31.028820 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:31.028820 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:14:31.028820 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:14:31.028820 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:14:31.028820 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:14:31.028820 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:14:31.028820 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:14:31.028820 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 02 09:14:31.028820 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:31.031360 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 322/658] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Sep 2 09:14:30 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:31.032411 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e38bc2-202f-41b0-9a30-ab6da6faa703 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:31.054026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc2eadf-dee2-412f-a42b-3992a36a3d70 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:31.061777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:31.091111 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363f713-47c2-4878-b7ea-f192829b54be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:31.091750 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363f713-47c2-4878-b7ea-f192829b54be None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:31.092030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363f713-47c2-4878-b7ea-f192829b54be None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:31.092304 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363f713-47c2-4878-b7ea-f192829b54be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:31.103866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363f713-47c2-4878-b7ea-f192829b54be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:31.105242 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363f713-47c2-4878-b7ea-f192829b54be None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:31.120344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-c363f713-47c2-4878-b7ea-f192829b54be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:31.120344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:31.120344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:31.120344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:31.120344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:31.120344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:31.120344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:31.120344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:31.120344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:31.120344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:31.120344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:31.120344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:31.120344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:31.120344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:31.120344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:31.120344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:31.121320 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:31.121320 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:31.121320 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:31.121320 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:31.121320 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:31.121320 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:31.121320 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:31.121320 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:31.121320 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:31.121320 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:31.121320 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:31.121320 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:31.121320 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:31.121320 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:31.121320 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:31.121320 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:31.121320 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:31.122185 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:31.122185 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:31.122185 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:31.122185 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 337/659] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:31.122873 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363f713-47c2-4878-b7ea-f192829b54be None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:31.128397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:31.145323 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e38bc2-202f-41b0-9a30-ab6da6faa703 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:31.182672 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3809a91-ef99-4135-b539-325de58aa34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:31.183689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3809a91-ef99-4135-b539-325de58aa34d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:31.184172 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3809a91-ef99-4135-b539-325de58aa34d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:31.184643 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3809a91-ef99-4135-b539-325de58aa34d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:31.228286 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363f713-47c2-4878-b7ea-f192829b54be None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:31.259404 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3809a91-ef99-4135-b539-325de58aa34d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:31.261238 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3809a91-ef99-4135-b539-325de58aa34d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:31.264886 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4b834-2e42-4ff8-bcf5-eca9d02c0870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:31.265331 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4b834-2e42-4ff8-bcf5-eca9d02c0870 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:31.265610 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4b834-2e42-4ff8-bcf5-eca9d02c0870 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:31.265883 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4b834-2e42-4ff8-bcf5-eca9d02c0870 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:31.266623 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4b834-2e42-4ff8-bcf5-eca9d02c0870 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:31.266964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4b834-2e42-4ff8-bcf5-eca9d02c0870 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:31.268404 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4b834-2e42-4ff8-bcf5-eca9d02c0870 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:31.287801 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 338/660] 10.222.0.57 () {56 vars in 1139 bytes} [Fri Sep 2 09:14:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:31.288681 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4b834-2e42-4ff8-bcf5-eca9d02c0870 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:31.307124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:31.353205 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:31.376748 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 323/661] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:14:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:14:31.377940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3809a91-ef99-4135-b539-325de58aa34d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:31.393386 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4b834-2e42-4ff8-bcf5-eca9d02c0870 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:31.431171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde93c2-d62b-4bef-9853-bdcc943e38f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:31.431951 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde93c2-d62b-4bef-9853-bdcc943e38f1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:31.432235 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde93c2-d62b-4bef-9853-bdcc943e38f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:31.432778 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde93c2-d62b-4bef-9853-bdcc943e38f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:31.433438 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde93c2-d62b-4bef-9853-bdcc943e38f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:31.433746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde93c2-d62b-4bef-9853-bdcc943e38f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:31.435128 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde93c2-d62b-4bef-9853-bdcc943e38f1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:31.450738 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 339/662] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:31.451580 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde93c2-d62b-4bef-9853-bdcc943e38f1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:31.485819 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3809a91-ef99-4135-b539-325de58aa34d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:31.521421 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5cce5-0bdf-4294-bd77-6d2d3bcb8812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:31.522024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5cce5-0bdf-4294-bd77-6d2d3bcb8812 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:31.522334 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5cce5-0bdf-4294-bd77-6d2d3bcb8812 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:31.522732 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5cce5-0bdf-4294-bd77-6d2d3bcb8812 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:31.533305 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:31.557719 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde93c2-d62b-4bef-9853-bdcc943e38f1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:31.579787 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:31.592671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f5cce5-0bdf-4294-bd77-6d2d3bcb8812 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:31.594107 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f5cce5-0bdf-4294-bd77-6d2d3bcb8812 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:31.595178 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b950236c-03ce-4d7a-8cad-61ccfb3c822f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:31.596127 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b950236c-03ce-4d7a-8cad-61ccfb3c822f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:31.596592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b950236c-03ce-4d7a-8cad-61ccfb3c822f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:31.596863 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b950236c-03ce-4d7a-8cad-61ccfb3c822f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:31.608176 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b950236c-03ce-4d7a-8cad-61ccfb3c822f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:31.609595 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b950236c-03ce-4d7a-8cad-61ccfb3c822f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:31.624009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-b950236c-03ce-4d7a-8cad-61ccfb3c822f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:31.624009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:31.624009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:31.624009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:31.624009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:31.624009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:31.624009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:31.624009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:31.624009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:31.624009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:31.624009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:31.624009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:31.624009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:31.624009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:31.624009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:31.624009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:31.625152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:31.625152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:31.625152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:31.625152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:31.625152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:31.625152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:31.625152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:31.625152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:31.625152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:31.625152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:31.625152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:31.625152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:31.625152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:31.625152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:31.625152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:31.625152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:31.625152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:31.626202 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:31.626202 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:31.626202 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:31.626202 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 340/663] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:31.626202 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f038c5f-413a-46ff-9f0b-c804408a4958 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:31.626483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f038c5f-413a-46ff-9f0b-c804408a4958 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:31.626745 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f038c5f-413a-46ff-9f0b-c804408a4958 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:31.628027 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 341/664] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:31.628657 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f038c5f-413a-46ff-9f0b-c804408a4958 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:31.699787 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 324/665] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:14:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:14:31.700893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c85f5-c62d-4e9e-85c9-babc2e41ab9d None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:31.701176 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c85f5-c62d-4e9e-85c9-babc2e41ab9d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:31.701445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c85f5-c62d-4e9e-85c9-babc2e41ab9d None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:31.702857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 325/666] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:31.703413 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c85f5-c62d-4e9e-85c9-babc2e41ab9d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:31.730065 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f038c5f-413a-46ff-9f0b-c804408a4958 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:31.756484 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:31.764381 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eb5045-2e47-4a17-9201-bc59bf050e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:31.764999 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb5045-2e47-4a17-9201-bc59bf050e2a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:31.765271 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb5045-2e47-4a17-9201-bc59bf050e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:31.765539 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eb5045-2e47-4a17-9201-bc59bf050e2a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:31.772027 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eb5045-2e47-4a17-9201-bc59bf050e2a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:31.773535 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85eb5045-2e47-4a17-9201-bc59bf050e2a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:31.784519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-85eb5045-2e47-4a17-9201-bc59bf050e2a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:31.784519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:31.784519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:31.784519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:31.784519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:31.784519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:31.784519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:31.784519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:31.784519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:31.784519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:31.784519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:31.784519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:31.784519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:14:31.784519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:14:31.784519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:14:31.785436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:14:31.785436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:31.785436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:31.785436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:31.785436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:31.785436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:31.785436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:31.785436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:31.785436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:31.785436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:31.785436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:31.785436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:31.785436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:31.785436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:31.785436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:31.785436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:31.785436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:31.786324 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:14:31.786324 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:14:31.786324 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:14:31.786324 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:14:31.786324 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:14:31.786324 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:14:31.786324 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:31.786324 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:31.786324 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 342/667] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:31.786833 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbc46d-5f42-413c-a77d-5d25a0145778 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:31.787004 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbc46d-5f42-413c-a77d-5d25a0145778 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:31.787270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbc46d-5f42-413c-a77d-5d25a0145778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:31.791676 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:31.806469 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c8dbc46d-5f42-413c-a77d-5d25a0145778 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:31.807642 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c85f5-c62d-4e9e-85c9-babc2e41ab9d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:31.807759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c8dbc46d-5f42-413c-a77d-5d25a0145778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:31.844785 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63c2de-0a7a-497a-8648-2fd3d8300e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:31.845399 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63c2de-0a7a-497a-8648-2fd3d8300e54 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:31.845685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63c2de-0a7a-497a-8648-2fd3d8300e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:31.845955 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63c2de-0a7a-497a-8648-2fd3d8300e54 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:31.846594 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63c2de-0a7a-497a-8648-2fd3d8300e54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:31.846923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63c2de-0a7a-497a-8648-2fd3d8300e54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:31.848284 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b63c2de-0a7a-497a-8648-2fd3d8300e54 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:31.860221 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 326/668] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:31.861425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63c2de-0a7a-497a-8648-2fd3d8300e54 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:31.886807 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 343/669] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:31.886807 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0f9ae-ce77-4de7-b4f7-f3eae26a04a9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:31.886807 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0f9ae-ce77-4de7-b4f7-f3eae26a04a9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:31.886807 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0f9ae-ce77-4de7-b4f7-f3eae26a04a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:31.895203 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-bfb0f9ae-ce77-4de7-b4f7-f3eae26a04a9 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:31.896384 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bfb0f9ae-ce77-4de7-b4f7-f3eae26a04a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:31.964480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 344/670] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:31.965318 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0f9ae-ce77-4de7-b4f7-f3eae26a04a9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:31.978641 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63c2de-0a7a-497a-8648-2fd3d8300e54 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:32.018438 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdbadc-d57c-4804-904c-492e09491f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:32.019244 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdbadc-d57c-4804-904c-492e09491f7c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:32.019615 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdbadc-d57c-4804-904c-492e09491f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:32.019982 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdbadc-d57c-4804-904c-492e09491f7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:32.025593 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:32.033430 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bdbadc-d57c-4804-904c-492e09491f7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:32.035004 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bdbadc-d57c-4804-904c-492e09491f7c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:32.041983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:32.050213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-15bdbadc-d57c-4804-904c-492e09491f7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:32.050213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:32.050213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:32.050213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:32.050213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:32.050213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:32.050213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:32.050213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:32.050213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:32.050213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:32.050213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:32.050213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:32.050213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:32.050213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:32.050213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:32.050213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:32.051239 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:32.051239 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:32.051239 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:32.051239 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:32.051239 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:32.051239 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:32.051239 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:32.051239 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:32.051239 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:32.051239 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:32.051239 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:32.051239 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:32.051239 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:32.051239 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:32.051239 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:32.051239 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:32.051239 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:32.052131 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:32.052131 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:32.052131 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:32.052131 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 327/671] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:32.052879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdbadc-d57c-4804-904c-492e09491f7c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:32.089757 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0f9ae-ce77-4de7-b4f7-f3eae26a04a9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:32.134343 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be343bfc-2dc0-4163-90aa-076f4fa35f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:32.136361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be343bfc-2dc0-4163-90aa-076f4fa35f5e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:32.136913 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be343bfc-2dc0-4163-90aa-076f4fa35f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:32.137737 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be343bfc-2dc0-4163-90aa-076f4fa35f5e None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:32.139040 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be343bfc-2dc0-4163-90aa-076f4fa35f5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:32.139826 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be343bfc-2dc0-4163-90aa-076f4fa35f5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:32.142288 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be343bfc-2dc0-4163-90aa-076f4fa35f5e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:32.154744 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 345/672] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:14:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:32.155897 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be343bfc-2dc0-4163-90aa-076f4fa35f5e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:32.164854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bdbadc-d57c-4804-904c-492e09491f7c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:32.201161 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed7359-cdb4-4e9e-8301-c058eb1f0de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:32.201894 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed7359-cdb4-4e9e-8301-c058eb1f0de3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:32.202172 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed7359-cdb4-4e9e-8301-c058eb1f0de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:32.202442 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed7359-cdb4-4e9e-8301-c058eb1f0de3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:32.210128 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed7359-cdb4-4e9e-8301-c058eb1f0de3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:32.211722 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed7359-cdb4-4e9e-8301-c058eb1f0de3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:32.223530 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-1aed7359-cdb4-4e9e-8301-c058eb1f0de3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:32.223530 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:32.223530 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:32.223530 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:32.223530 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:32.223530 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:32.223530 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:32.223530 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:32.223530 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:32.223530 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:32.223530 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:32.223530 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:32.223530 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:14:32.223530 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:14:32.223530 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:14:32.224379 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:14:32.224379 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:32.224379 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:32.224379 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:32.224379 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:32.224379 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:32.224379 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:32.224379 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:32.224379 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:32.224379 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:32.224379 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:32.224379 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:32.224379 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:32.224379 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:32.224379 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:32.224379 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:32.224379 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:32.225869 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:14:32.225869 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:14:32.225869 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:14:32.225869 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:14:32.225869 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:14:32.225869 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:14:32.225869 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:32.225869 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:32.225869 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 328/673] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:32.226456 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed7359-cdb4-4e9e-8301-c058eb1f0de3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:32.263129 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be343bfc-2dc0-4163-90aa-076f4fa35f5e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:32.271741 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:32.285308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:32.296882 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365eb718-3097-42fe-a71a-5ebaf1c1757a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:32.297486 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365eb718-3097-42fe-a71a-5ebaf1c1757a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:32.297764 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365eb718-3097-42fe-a71a-5ebaf1c1757a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:32.298032 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365eb718-3097-42fe-a71a-5ebaf1c1757a None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:32.298668 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365eb718-3097-42fe-a71a-5ebaf1c1757a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:32.299002 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365eb718-3097-42fe-a71a-5ebaf1c1757a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:32.300354 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365eb718-3097-42fe-a71a-5ebaf1c1757a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:32.312472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 346/674] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:32.313268 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365eb718-3097-42fe-a71a-5ebaf1c1757a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:32.326068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed7359-cdb4-4e9e-8301-c058eb1f0de3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:32.360556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc93304-924b-4050-937e-2bac27205ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:32.361182 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc93304-924b-4050-937e-2bac27205ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:32.361464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc93304-924b-4050-937e-2bac27205ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:32.361744 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc93304-924b-4050-937e-2bac27205ec5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:32.372106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc93304-924b-4050-937e-2bac27205ec5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:32.377267 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-bfc93304-924b-4050-937e-2bac27205ec5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:14:32.377267 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:32.377267 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:32.377267 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:32.377267 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:32.377267 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:32.377267 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:32.377267 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:32.377267 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:32.377267 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:32.377267 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:32.377267 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:32.377267 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:32.377267 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:32.377267 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:32.377267 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:32.378169 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:32.378169 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:32.378169 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:32.378169 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:32.378169 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:32.378169 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:32.378169 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:32.378169 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:32.378169 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:32.378169 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:32.378169 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:32.378169 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:32.378169 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:32.378169 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:32.378169 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:32.378169 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:32.379056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:32.379056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:32.379056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:32.379056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:32.379056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:32.379056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:32.379056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:32.379056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:32.379056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:32.379056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:32.379056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:14:32.379056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:32.379056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 329/675] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:14:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:32.380376 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc93304-924b-4050-937e-2bac27205ec5 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:32.419623 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365eb718-3097-42fe-a71a-5ebaf1c1757a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:32.452638 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9627c64-69d8-49d4-b067-53c9e4322b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:32.453239 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9627c64-69d8-49d4-b067-53c9e4322b29 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:32.453524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9627c64-69d8-49d4-b067-53c9e4322b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:32.453797 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9627c64-69d8-49d4-b067-53c9e4322b29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:32.454520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9627c64-69d8-49d4-b067-53c9e4322b29 None admin] RBAC: Adding query filter params (role.id=faf1138f11d34abe8a862f1a1f7c140a, scope.project.id=d18a5117a03b4f21ab3290a15680de66, user.id=b6a1f44f6fe94c09b696867871fd5b47) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:32.454859 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9627c64-69d8-49d4-b067-53c9e4322b29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:32.456377 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9627c64-69d8-49d4-b067-53c9e4322b29 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:32.462553 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 347/676] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:14:32 2022] GET /identity/v3/role_assignments?user.id=b6a1f44f6fe94c09b696867871fd5b47&scope.project.id=d18a5117a03b4f21ab3290a15680de66&role.id=faf1138f11d34abe8a862f1a1f7c140a => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:32.463367 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9627c64-69d8-49d4-b067-53c9e4322b29 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:32.486209 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc93304-924b-4050-937e-2bac27205ec5 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:32.499782 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:32.505307 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:32.524187 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb066b8f-1d72-47e1-ac04-ab988d0353f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:32.524821 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb066b8f-1d72-47e1-ac04-ab988d0353f6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:32.525096 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb066b8f-1d72-47e1-ac04-ab988d0353f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:32.525369 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb066b8f-1d72-47e1-ac04-ab988d0353f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:32.526232 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb066b8f-1d72-47e1-ac04-ab988d0353f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:32.526539 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb066b8f-1d72-47e1-ac04-ab988d0353f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:32.528049 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb066b8f-1d72-47e1-ac04-ab988d0353f6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:32.542194 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 330/677] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:14:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:32.543545 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb066b8f-1d72-47e1-ac04-ab988d0353f6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:32.588853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9627c64-69d8-49d4-b067-53c9e4322b29 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:32.624364 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f52eb-48d3-47f8-a7db-c7d9c4536f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:32.624995 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f52eb-48d3-47f8-a7db-c7d9c4536f94 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:32.625271 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f52eb-48d3-47f8-a7db-c7d9c4536f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:32.626330 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f52eb-48d3-47f8-a7db-c7d9c4536f94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:32.638530 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f52eb-48d3-47f8-a7db-c7d9c4536f94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:32.644968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-df4f52eb-48d3-47f8-a7db-c7d9c4536f94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:32.644968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:32.644968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:32.644968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:32.644968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:32.644968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:32.644968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:32.644968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:32.644968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:32.644968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:32.644968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:32.644968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:32.644968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:32.644968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:32.644968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:32.644968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:32.645948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:32.645948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:32.645948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:32.645948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:32.645948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:32.645948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:32.645948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:32.645948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:32.645948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:32.645948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:32.645948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:32.645948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:32.645948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:32.645948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:32.645948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:32.645948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:32.646802 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:32.646802 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:32.646802 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:32.646802 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:32.646802 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:32.646802 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:32.646802 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:32.646802 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:32.646802 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:32.646802 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:32.646802 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:32.646802 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:32.649288 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 348/678] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Sep 2 09:14:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:32.649288 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f52eb-48d3-47f8-a7db-c7d9c4536f94 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:32.667402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb066b8f-1d72-47e1-ac04-ab988d0353f6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:32.704041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55a19d-e0ad-41b9-ac4c-f83cd4a91d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:32.704602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55a19d-e0ad-41b9-ac4c-f83cd4a91d18 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:32.704882 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55a19d-e0ad-41b9-ac4c-f83cd4a91d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:32.705158 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c55a19d-e0ad-41b9-ac4c-f83cd4a91d18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:32.712675 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55a19d-e0ad-41b9-ac4c-f83cd4a91d18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:32.714096 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c55a19d-e0ad-41b9-ac4c-f83cd4a91d18 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:32.726357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-6c55a19d-e0ad-41b9-ac4c-f83cd4a91d18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:32.726357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:32.726357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:32.726357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:32.726357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:32.726357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:32.726357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:32.726357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:32.726357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:32.726357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:32.726357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:32.726357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:32.726357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:14:32.726357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:14:32.726357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:14:32.727487 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:14:32.727487 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:32.727487 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:32.727487 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:32.727487 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:32.727487 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:32.727487 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:32.727487 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:32.727487 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:32.727487 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:32.727487 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:32.727487 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:32.727487 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:32.727487 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:32.727487 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:32.727487 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:32.727487 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:32.728704 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:14:32.728704 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:14:32.728704 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:14:32.728704 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:14:32.728704 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:14:32.728704 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:14:32.728704 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:32.728704 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:32.732136 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 331/679] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:32.732855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55a19d-e0ad-41b9-ac4c-f83cd4a91d18 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:32.749236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:32.758819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f52eb-48d3-47f8-a7db-c7d9c4536f94 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:32.773009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:32.795334 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8fad21-640f-4920-a4e9-174fd30c47ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:32.795940 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8fad21-640f-4920-a4e9-174fd30c47ac None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:32.796219 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8fad21-640f-4920-a4e9-174fd30c47ac None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:32.796490 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8fad21-640f-4920-a4e9-174fd30c47ac None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:32.797148 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8fad21-640f-4920-a4e9-174fd30c47ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:32.797447 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8fad21-640f-4920-a4e9-174fd30c47ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:32.798821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8fad21-640f-4920-a4e9-174fd30c47ac None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:32.811873 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 349/680] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:14:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:32.811873 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8fad21-640f-4920-a4e9-174fd30c47ac None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:32.838617 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c55a19d-e0ad-41b9-ac4c-f83cd4a91d18 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:32.876429 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13673744-b6df-495a-9ea0-b51fe99aaaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:32.877034 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13673744-b6df-495a-9ea0-b51fe99aaaa6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:32.877308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13673744-b6df-495a-9ea0-b51fe99aaaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:32.877574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13673744-b6df-495a-9ea0-b51fe99aaaa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:32.878295 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13673744-b6df-495a-9ea0-b51fe99aaaa6 None admin] RBAC: Adding query filter params (role.id=faf1138f11d34abe8a862f1a1f7c140a, scope.project.id=d18a5117a03b4f21ab3290a15680de66, user.id=31b555bb94ce44ce918cbf2a012576a7) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:32.878601 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13673744-b6df-495a-9ea0-b51fe99aaaa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:32.880041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13673744-b6df-495a-9ea0-b51fe99aaaa6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:32.894735 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 332/681] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:14:32 2022] GET /identity/v3/role_assignments?user.id=31b555bb94ce44ce918cbf2a012576a7&scope.project.id=d18a5117a03b4f21ab3290a15680de66&role.id=faf1138f11d34abe8a862f1a1f7c140a => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:32.894735 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13673744-b6df-495a-9ea0-b51fe99aaaa6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:32.936862 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8fad21-640f-4920-a4e9-174fd30c47ac None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:32.977386 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999de2b6-54e0-4472-b0c9-8e153f179ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:32.978099 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999de2b6-54e0-4472-b0c9-8e153f179ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:32.978451 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999de2b6-54e0-4472-b0c9-8e153f179ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:32.978841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999de2b6-54e0-4472-b0c9-8e153f179ac8 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:32.979633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999de2b6-54e0-4472-b0c9-8e153f179ac8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:32.980005 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999de2b6-54e0-4472-b0c9-8e153f179ac8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:32.981467 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999de2b6-54e0-4472-b0c9-8e153f179ac8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:32.991144 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 350/682] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Sep 2 09:14:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:32.992130 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999de2b6-54e0-4472-b0c9-8e153f179ac8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:33.012499 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:33.022004 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13673744-b6df-495a-9ea0-b51fe99aaaa6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:33.025116 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:33.076385 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3f7e4-225c-4b3c-8587-fa94a3152e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:33.077362 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3f7e4-225c-4b3c-8587-fa94a3152e63 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:33.077362 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3f7e4-225c-4b3c-8587-fa94a3152e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:33.077643 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3f7e4-225c-4b3c-8587-fa94a3152e63 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:33.078406 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f3f7e4-225c-4b3c-8587-fa94a3152e63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:33.078595 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f3f7e4-225c-4b3c-8587-fa94a3152e63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:33.081275 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f3f7e4-225c-4b3c-8587-fa94a3152e63 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:33.093562 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 333/683] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:14:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:33.094413 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3f7e4-225c-4b3c-8587-fa94a3152e63 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:33.118854 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999de2b6-54e0-4472-b0c9-8e153f179ac8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:33.159565 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa7974-c3b1-4d52-bfd9-f999d75a22b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:33.160193 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa7974-c3b1-4d52-bfd9-f999d75a22b1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:33.160468 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa7974-c3b1-4d52-bfd9-f999d75a22b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:33.160743 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa7974-c3b1-4d52-bfd9-f999d75a22b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:33.173661 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfa7974-c3b1-4d52-bfd9-f999d75a22b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:33.175433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfa7974-c3b1-4d52-bfd9-f999d75a22b1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:33.191584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4dfa7974-c3b1-4d52-bfd9-f999d75a22b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:33.191584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:33.191584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:33.191584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:33.191584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:33.191584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:33.191584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:33.191584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:33.191584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:33.191584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:33.191584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:33.191584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:33.191584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:33.191584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:33.191584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:33.191584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:33.192545 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:33.192545 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:33.192545 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:33.192545 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:33.192545 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:33.192545 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:33.192545 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:33.192545 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:33.192545 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:33.192545 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:33.192545 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:33.192545 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:33.192545 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:33.192545 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:33.192545 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:33.192545 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:33.192545 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:33.193421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:33.193421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:33.193421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:33.193421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 351/684] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:33.196569 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa7974-c3b1-4d52-bfd9-f999d75a22b1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:33.198735 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3f7e4-225c-4b3c-8587-fa94a3152e63 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:33.235529 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393b494-e4db-49b8-b2fb-10508da42056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:33.236153 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393b494-e4db-49b8-b2fb-10508da42056 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:33.236406 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393b494-e4db-49b8-b2fb-10508da42056 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:33.236679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393b494-e4db-49b8-b2fb-10508da42056 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:33.247984 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7393b494-e4db-49b8-b2fb-10508da42056 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:33.249581 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7393b494-e4db-49b8-b2fb-10508da42056 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:33.267456 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-7393b494-e4db-49b8-b2fb-10508da42056 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:33.267456 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:33.267456 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:33.267456 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:33.267456 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:33.267456 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:33.267456 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:33.267456 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:33.267456 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:33.267456 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:33.267456 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:33.267456 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:33.267456 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:33.267456 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:33.267456 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:33.267456 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:33.268437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:33.268437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:33.268437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:33.268437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:33.268437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:33.268437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:33.268437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:33.268437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:33.268437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:33.268437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:33.268437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:33.268437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:33.268437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:33.268437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:33.268437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:33.268437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:33.268437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:33.269319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:33.269319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:33.269319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:33.269319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 334/685] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:33.269961 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393b494-e4db-49b8-b2fb-10508da42056 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:33.278088 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:33.286723 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:33.314460 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa7974-c3b1-4d52-bfd9-f999d75a22b1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:33.352266 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b25c6e-5578-48ea-92bd-d59eea160568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:33.353379 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b25c6e-5578-48ea-92bd-d59eea160568 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:33.353904 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b25c6e-5578-48ea-92bd-d59eea160568 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:33.354485 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b25c6e-5578-48ea-92bd-d59eea160568 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:33.355549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b25c6e-5578-48ea-92bd-d59eea160568 None admin] RBAC: Adding query filter params (role.id=faf1138f11d34abe8a862f1a1f7c140a, scope.project.id=d18a5117a03b4f21ab3290a15680de66, user.id=dfea2ee0c9e84b48ac9bafe3173d937a) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:33.356057 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b25c6e-5578-48ea-92bd-d59eea160568 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:33.357734 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b25c6e-5578-48ea-92bd-d59eea160568 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:33.364650 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 352/686] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:14:33 2022] GET /identity/v3/role_assignments?user.id=dfea2ee0c9e84b48ac9bafe3173d937a&scope.project.id=d18a5117a03b4f21ab3290a15680de66&role.id=faf1138f11d34abe8a862f1a1f7c140a => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:33.365782 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b25c6e-5578-48ea-92bd-d59eea160568 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:33.379000 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393b494-e4db-49b8-b2fb-10508da42056 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:33.419050 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e92b7eb-662c-4d89-a0c4-337c2accfdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:33.419706 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92b7eb-662c-4d89-a0c4-337c2accfdaa None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:33.420034 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92b7eb-662c-4d89-a0c4-337c2accfdaa None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:33.420306 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e92b7eb-662c-4d89-a0c4-337c2accfdaa None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:33.420966 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e92b7eb-662c-4d89-a0c4-337c2accfdaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:33.421263 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e92b7eb-662c-4d89-a0c4-337c2accfdaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:33.422897 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e92b7eb-662c-4d89-a0c4-337c2accfdaa None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:33.436057 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 335/687] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:33.442356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e92b7eb-662c-4d89-a0c4-337c2accfdaa None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:33.529919 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b25c6e-5578-48ea-92bd-d59eea160568 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:33.587092 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:33.592018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:33.593710 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efcce1a-5e7b-4098-8e7b-6ad386b3d2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:33.594569 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efcce1a-5e7b-4098-8e7b-6ad386b3d2a2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:33.595045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efcce1a-5e7b-4098-8e7b-6ad386b3d2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:33.595538 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efcce1a-5e7b-4098-8e7b-6ad386b3d2a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:33.596351 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efcce1a-5e7b-4098-8e7b-6ad386b3d2a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:33.599730 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efcce1a-5e7b-4098-8e7b-6ad386b3d2a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:33.599730 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efcce1a-5e7b-4098-8e7b-6ad386b3d2a2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:33.612923 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 353/688] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:33.623764 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efcce1a-5e7b-4098-8e7b-6ad386b3d2a2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:33.632270 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e92b7eb-662c-4d89-a0c4-337c2accfdaa None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:33.671655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9670f38-e2cd-414e-8a30-8f105762d080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:33.672659 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9670f38-e2cd-414e-8a30-8f105762d080 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:33.673141 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9670f38-e2cd-414e-8a30-8f105762d080 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:33.673624 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9670f38-e2cd-414e-8a30-8f105762d080 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:33.682029 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9670f38-e2cd-414e-8a30-8f105762d080 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:33.683739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9670f38-e2cd-414e-8a30-8f105762d080 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:33.696475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-a9670f38-e2cd-414e-8a30-8f105762d080 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 02 09:14:33.696475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:33.696475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:33.696475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:33.696475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:33.696475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:33.696475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:33.696475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:33.696475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:33.696475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:33.696475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:33.696475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:33.696475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:14:33.696475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:14:33.696475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:14:33.696475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:14:33.697402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:33.697402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:33.697402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:14:33.697402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:33.697402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:14:33.697402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:33.697402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:33.697402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:33.697402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:33.697402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:33.697402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:33.697402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:33.697402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:33.697402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:33.697402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:33.697402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:33.697402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:33.697402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:33.698604 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:33.698604 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:33.698604 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:14:33.698604 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:14:33.698604 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:14:33.698604 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:14:33.698604 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:14:33.698604 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:14:33.698604 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 02 09:14:33.698604 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:33.701079 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 336/689] 10.222.0.57 () {56 vars in 1093 bytes} [Fri Sep 2 09:14:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:33.706236 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9670f38-e2cd-414e-8a30-8f105762d080 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:33.750074 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efcce1a-5e7b-4098-8e7b-6ad386b3d2a2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:33.791501 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75ad73-b1ed-45fd-bdee-25fb5dcdc956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:33.792057 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75ad73-b1ed-45fd-bdee-25fb5dcdc956 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:33.792340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75ad73-b1ed-45fd-bdee-25fb5dcdc956 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:33.792614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75ad73-b1ed-45fd-bdee-25fb5dcdc956 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:33.800210 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d75ad73-b1ed-45fd-bdee-25fb5dcdc956 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:33.801669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d75ad73-b1ed-45fd-bdee-25fb5dcdc956 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:33.813861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-5d75ad73-b1ed-45fd-bdee-25fb5dcdc956 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 02 09:14:33.813861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:33.813861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:33.813861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:33.813861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:33.813861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:33.813861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:33.813861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:33.813861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:33.813861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:33.813861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:33.813861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:33.813861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:14:33.813861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:14:33.813861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:14:33.813861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:14:33.814833 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:33.814833 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:33.814833 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:14:33.814833 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:33.814833 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:14:33.814833 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:33.814833 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:33.814833 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:33.814833 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:33.814833 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:33.814833 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:33.814833 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:33.814833 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:33.814833 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:33.814833 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:33.814833 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:33.814833 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:33.814833 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:33.815779 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:33.815779 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:33.815779 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:14:33.815779 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:14:33.815779 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:14:33.815779 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:14:33.815779 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:14:33.815779 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:14:33.815779 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 02 09:14:33.815779 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:33.815779 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 354/690] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:14:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:33.821475 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75ad73-b1ed-45fd-bdee-25fb5dcdc956 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:33.838880 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9670f38-e2cd-414e-8a30-8f105762d080 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:33.883534 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:33.920347 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27423c74-561f-4d7f-85da-8cb40dbe77c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:33.920970 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27423c74-561f-4d7f-85da-8cb40dbe77c2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:33.921252 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27423c74-561f-4d7f-85da-8cb40dbe77c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:33.921522 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27423c74-561f-4d7f-85da-8cb40dbe77c2 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:33.922252 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27423c74-561f-4d7f-85da-8cb40dbe77c2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:33.922553 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27423c74-561f-4d7f-85da-8cb40dbe77c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:33.924003 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27423c74-561f-4d7f-85da-8cb40dbe77c2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:33.926761 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:33.929997 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75ad73-b1ed-45fd-bdee-25fb5dcdc956 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:33.966322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce32f2ef-b035-451a-b6c8-37a3ac1840ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:33.966962 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32f2ef-b035-451a-b6c8-37a3ac1840ed None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:33.967351 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 337/691] 10.222.0.57 () {56 vars in 1133 bytes} [Fri Sep 2 09:14:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:33.967462 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32f2ef-b035-451a-b6c8-37a3ac1840ed None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:33.967641 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32f2ef-b035-451a-b6c8-37a3ac1840ed None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:33.968256 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27423c74-561f-4d7f-85da-8cb40dbe77c2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:33.968427 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce32f2ef-b035-451a-b6c8-37a3ac1840ed None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:33.968726 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce32f2ef-b035-451a-b6c8-37a3ac1840ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:33.970147 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce32f2ef-b035-451a-b6c8-37a3ac1840ed None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:33.989601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 355/692] 10.222.0.57 () {56 vars in 1135 bytes} [Fri Sep 2 09:14:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:14:33.995738 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce32f2ef-b035-451a-b6c8-37a3ac1840ed None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:34.097361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce32f2ef-b035-451a-b6c8-37a3ac1840ed None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:34.119078 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:34.149718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27423c74-561f-4d7f-85da-8cb40dbe77c2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:34.167971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ba4d6-1ded-4cc1-940e-f5ff94c4cbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:34.168651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ba4d6-1ded-4cc1-940e-f5ff94c4cbdd None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:34.168931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ba4d6-1ded-4cc1-940e-f5ff94c4cbdd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:34.169212 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ba4d6-1ded-4cc1-940e-f5ff94c4cbdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:34.195442 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9fb43a-8c33-441a-b30c-f84f42526bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:34.196058 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fb43a-8c33-441a-b30c-f84f42526bfd None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:34.196335 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fb43a-8c33-441a-b30c-f84f42526bfd None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:34.196607 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fb43a-8c33-441a-b30c-f84f42526bfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:34.309754 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ba4d6-1ded-4cc1-940e-f5ff94c4cbdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:34.311257 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ba4d6-1ded-4cc1-940e-f5ff94c4cbdd None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:34.335717 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:34.345476 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9fb43a-8c33-441a-b30c-f84f42526bfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:34.346946 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9fb43a-8c33-441a-b30c-f84f42526bfd None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:34.504482 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 338/693] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:14:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:14:34.505669 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e0d52-cd96-4842-8e91-4d8f715ffaec None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:34.505952 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e0d52-cd96-4842-8e91-4d8f715ffaec None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:34.506222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7e0d52-cd96-4842-8e91-4d8f715ffaec None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:34.508298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 339/694] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:34.514487 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59765c-7551-4dab-840a-2c6e4d8f053f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:34.514804 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59765c-7551-4dab-840a-2c6e4d8f053f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:34.515081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c59765c-7551-4dab-840a-2c6e4d8f053f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:34.530616 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 356/695] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:14:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 02 09:14:34.531611 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ba4d6-1ded-4cc1-940e-f5ff94c4cbdd None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:34.538438 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3c59765c-7551-4dab-840a-2c6e4d8f053f None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:34.540392 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3c59765c-7551-4dab-840a-2c6e4d8f053f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:34.543704 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:34.642840 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 340/696] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:34.644154 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c59765c-7551-4dab-840a-2c6e4d8f053f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:34.650602 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ba4d6-1ded-4cc1-940e-f5ff94c4cbdd None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:34.689767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc47b6-2649-4d60-94e9-38238dc12055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:34.690904 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc47b6-2649-4d60-94e9-38238dc12055 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:34.691384 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc47b6-2649-4d60-94e9-38238dc12055 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:34.691872 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc47b6-2649-4d60-94e9-38238dc12055 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:34.697452 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:34.704961 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc47b6-2649-4d60-94e9-38238dc12055 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:34.706644 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc47b6-2649-4d60-94e9-38238dc12055 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:34.723455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-95bc47b6-2649-4d60-94e9-38238dc12055 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:34.723455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:34.723455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:34.723455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:34.723455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:34.723455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:34.723455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:34.723455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:34.723455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:34.723455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:34.723455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:34.723455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:34.723455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:34.723455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:34.723455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:34.723455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:34.724422 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:34.724422 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:34.724422 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:34.724422 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:34.724422 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:34.724422 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:34.724422 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:34.724422 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:34.724422 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:34.724422 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:34.724422 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:34.724422 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:34.724422 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:34.724422 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:34.724422 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:34.724422 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:34.724422 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:34.725369 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:34.725369 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:34.725369 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:34.727519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 357/697] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:34.734151 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc47b6-2649-4d60-94e9-38238dc12055 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:34.827161 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:34.854924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c59765c-7551-4dab-840a-2c6e4d8f053f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:34.874286 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc47b6-2649-4d60-94e9-38238dc12055 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:34.924817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408ef21-ff07-457a-9245-8dc41c26e9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:34.926181 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408ef21-ff07-457a-9245-8dc41c26e9ce None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:34.926531 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408ef21-ff07-457a-9245-8dc41c26e9ce None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:34.926833 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408ef21-ff07-457a-9245-8dc41c26e9ce None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:34.927495 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4408ef21-ff07-457a-9245-8dc41c26e9ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:34.927803 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4408ef21-ff07-457a-9245-8dc41c26e9ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:34.929211 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4408ef21-ff07-457a-9245-8dc41c26e9ce None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:34.944926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12add93-2447-4c7e-b485-661514bcfd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:34.945567 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12add93-2447-4c7e-b485-661514bcfd3e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:34.945847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12add93-2447-4c7e-b485-661514bcfd3e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:34.946752 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12add93-2447-4c7e-b485-661514bcfd3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:34.957088 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 358/698] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:34.958061 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408ef21-ff07-457a-9245-8dc41c26e9ce None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:34.965982 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12add93-2447-4c7e-b485-661514bcfd3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:34.967480 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12add93-2447-4c7e-b485-661514bcfd3e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:34.983155 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-f12add93-2447-4c7e-b485-661514bcfd3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:34.983155 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:34.983155 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:34.983155 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:34.983155 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:34.983155 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:34.983155 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:34.983155 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:34.983155 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:34.983155 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:34.983155 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:34.983155 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:34.983155 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:34.983155 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:34.983155 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:34.983155 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:34.984123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:34.984123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:34.984123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:34.984123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:34.984123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:34.984123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:34.984123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:34.984123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:34.984123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:34.984123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:34.984123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:34.984123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:34.984123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:34.984123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:34.984123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:34.984123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:34.984123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:34.985000 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:34.985000 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:34.985000 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:34.985000 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 341/699] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:34.985650 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12add93-2447-4c7e-b485-661514bcfd3e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:35.062715 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408ef21-ff07-457a-9245-8dc41c26e9ce None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:35.077283 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:35.088768 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:35.100880 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120bdc6-8487-45df-af28-933f0d8b7bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:35.101909 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120bdc6-8487-45df-af28-933f0d8b7bc3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:35.102774 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120bdc6-8487-45df-af28-933f0d8b7bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:35.103463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a120bdc6-8487-45df-af28-933f0d8b7bc3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:35.104192 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120bdc6-8487-45df-af28-933f0d8b7bc3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:35.105275 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12add93-2447-4c7e-b485-661514bcfd3e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:35.105925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a120bdc6-8487-45df-af28-933f0d8b7bc3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:35.120541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-a120bdc6-8487-45df-af28-933f0d8b7bc3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 02 09:14:35.120541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:35.120541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 09:14:35.120541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 09:14:35.120541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 09:14:35.120541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 09:14:35.120541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 09:14:35.120541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:14:35.120541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 02 09:14:35.120541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:35.120541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:14:35.120541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:35.120541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:35.120541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:35.120541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:35.120541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:35.120541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:35.121825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:35.121825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:35.121825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:35.121825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:35.121825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:35.121825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:35.121825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 09:14:35.121825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 09:14:35.121825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 09:14:35.121825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 09:14:35.121825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:35.121825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:35.121825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:35.121825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:35.121825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:35.121825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:35.122860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:35.122860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:35.122860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:35.122860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:35.122860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:35.122860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:35.122860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:35.122860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:35.122860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:35.122860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:35.122860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 09:14:35.122860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:14:35.122860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 02 09:14:35.122860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:35.128922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 359/700] 10.222.0.57 () {56 vars in 1129 bytes} [Fri Sep 2 09:14:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:35.128922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120bdc6-8487-45df-af28-933f0d8b7bc3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:35.150230 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911eb66a-32ea-4be0-acd1-d5430b5fc319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:35.150926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911eb66a-32ea-4be0-acd1-d5430b5fc319 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:35.151163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911eb66a-32ea-4be0-acd1-d5430b5fc319 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:35.151441 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911eb66a-32ea-4be0-acd1-d5430b5fc319 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:35.158332 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911eb66a-32ea-4be0-acd1-d5430b5fc319 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:35.160332 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911eb66a-32ea-4be0-acd1-d5430b5fc319 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:35.173789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-911eb66a-32ea-4be0-acd1-d5430b5fc319 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:35.173789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:35.173789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:35.173789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:35.173789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:35.173789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:35.173789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:35.173789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:35.173789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:35.173789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:35.173789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:35.173789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:35.173789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:14:35.173789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:14:35.173789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:14:35.176281 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:14:35.176281 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:35.176281 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:35.176281 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:35.176281 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:35.176281 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:35.176281 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:35.176281 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:35.176281 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:35.176281 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:35.176281 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:35.176281 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:35.176281 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:35.176281 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:35.176281 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:35.176281 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:35.176281 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:35.177381 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:14:35.177381 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:14:35.177381 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:14:35.177381 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:14:35.177381 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:14:35.177381 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:14:35.177381 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:35.177381 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:35.180784 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 342/701] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:35.181768 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911eb66a-32ea-4be0-acd1-d5430b5fc319 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:35.259986 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a120bdc6-8487-45df-af28-933f0d8b7bc3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:35.298090 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b252c39-18bb-47ab-92b0-55abee8f9ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:35.298739 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b252c39-18bb-47ab-92b0-55abee8f9ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:35.299006 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b252c39-18bb-47ab-92b0-55abee8f9ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:35.299277 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b252c39-18bb-47ab-92b0-55abee8f9ca7 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:35.299925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b252c39-18bb-47ab-92b0-55abee8f9ca7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:35.300227 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b252c39-18bb-47ab-92b0-55abee8f9ca7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:35.301597 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b252c39-18bb-47ab-92b0-55abee8f9ca7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:35.305816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911eb66a-32ea-4be0-acd1-d5430b5fc319 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:35.318101 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 360/702] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:35.318101 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b252c39-18bb-47ab-92b0-55abee8f9ca7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:35.347966 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973fe0d-2d80-4c62-b34c-fd022ca6fe3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:35.348655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973fe0d-2d80-4c62-b34c-fd022ca6fe3c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:35.348943 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973fe0d-2d80-4c62-b34c-fd022ca6fe3c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:35.349235 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973fe0d-2d80-4c62-b34c-fd022ca6fe3c None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:35.350014 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b973fe0d-2d80-4c62-b34c-fd022ca6fe3c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:35.350844 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b973fe0d-2d80-4c62-b34c-fd022ca6fe3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:35.352335 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b973fe0d-2d80-4c62-b34c-fd022ca6fe3c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:35.353562 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:35.362886 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 343/703] 10.222.0.57 () {56 vars in 1124 bytes} [Fri Sep 2 09:14:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:35.363065 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:35.372729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f97939-bc87-407a-87c4-40b0b889f645 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:35.372729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f97939-bc87-407a-87c4-40b0b889f645 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:35.372729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f97939-bc87-407a-87c4-40b0b889f645 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:35.372729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 344/704] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:35.372729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f97939-bc87-407a-87c4-40b0b889f645 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:35.423066 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b252c39-18bb-47ab-92b0-55abee8f9ca7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:35.459699 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8bb5ba-3c1b-42cb-beac-e8abecc14e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:35.460315 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bb5ba-3c1b-42cb-beac-e8abecc14e93 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:35.460598 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bb5ba-3c1b-42cb-beac-e8abecc14e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:35.460872 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bb5ba-3c1b-42cb-beac-e8abecc14e93 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:35.461553 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8bb5ba-3c1b-42cb-beac-e8abecc14e93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:35.461859 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8bb5ba-3c1b-42cb-beac-e8abecc14e93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:35.463315 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8bb5ba-3c1b-42cb-beac-e8abecc14e93 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:35.467314 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f97939-bc87-407a-87c4-40b0b889f645 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:35.480511 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 361/705] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:14:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:35.480511 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8bb5ba-3c1b-42cb-beac-e8abecc14e93 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:35.507845 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b533e07c-8645-4f2a-8869-bc486b7017d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:35.508472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533e07c-8645-4f2a-8869-bc486b7017d1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:35.508746 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533e07c-8645-4f2a-8869-bc486b7017d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:35.509020 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b533e07c-8645-4f2a-8869-bc486b7017d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:35.517748 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b533e07c-8645-4f2a-8869-bc486b7017d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:35.519752 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b533e07c-8645-4f2a-8869-bc486b7017d1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:35.532262 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-b533e07c-8645-4f2a-8869-bc486b7017d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:35.532262 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:35.532262 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:35.532262 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:35.532262 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:35.532262 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:35.532262 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:35.532262 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:35.532262 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:35.532262 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:35.532262 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:35.532262 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:35.532262 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:14:35.532262 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:14:35.532262 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:14:35.533419 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:14:35.533419 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:35.533419 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:35.533419 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:35.533419 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:35.533419 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:35.533419 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:35.533419 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:35.533419 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:35.533419 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:35.533419 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:35.533419 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:35.533419 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:35.533419 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:35.533419 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:35.533419 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:35.533419 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:35.534561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:14:35.534561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:14:35.534561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:14:35.534561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:14:35.534561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:14:35.534561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:14:35.534561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:35.534561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:35.534561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 345/706] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:35.535113 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5ca73-a55e-4733-bbb9-eb2be1ebc526 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:35.535113 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5ca73-a55e-4733-bbb9-eb2be1ebc526 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:35.535350 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db5ca73-a55e-4733-bbb9-eb2be1ebc526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:35.555967 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5db5ca73-a55e-4733-bbb9-eb2be1ebc526 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:35.557448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5db5ca73-a55e-4733-bbb9-eb2be1ebc526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:35.595089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8bb5ba-3c1b-42cb-beac-e8abecc14e93 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:35.602891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:35.613153 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:35.629707 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 346/707] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:35.630801 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db5ca73-a55e-4733-bbb9-eb2be1ebc526 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:35.644553 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f5e033-8ad1-45ad-aa37-df0222e53608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:35.645337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5e033-8ad1-45ad-aa37-df0222e53608 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:35.645725 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5e033-8ad1-45ad-aa37-df0222e53608 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:35.646268 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f5e033-8ad1-45ad-aa37-df0222e53608 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:35.647274 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f5e033-8ad1-45ad-aa37-df0222e53608 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:35.647583 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f5e033-8ad1-45ad-aa37-df0222e53608 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:35.649769 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f5e033-8ad1-45ad-aa37-df0222e53608 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:35.658939 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 362/708] 10.222.0.57 () {56 vars in 1124 bytes} [Fri Sep 2 09:14:35 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:35.659971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f5e033-8ad1-45ad-aa37-df0222e53608 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:35.765258 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db5ca73-a55e-4733-bbb9-eb2be1ebc526 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:35.802864 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef77951-cb74-4198-8519-26ecfe407fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:35.803891 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef77951-cb74-4198-8519-26ecfe407fcb None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:35.804325 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef77951-cb74-4198-8519-26ecfe407fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:35.804744 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef77951-cb74-4198-8519-26ecfe407fcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:35.805644 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef77951-cb74-4198-8519-26ecfe407fcb None admin] RBAC: Adding query filter params (role.id=f886b8e8889549e896a8fb3426cce9e1, scope.project.id=d18a5117a03b4f21ab3290a15680de66, user.id=1da355c8be2d496bb08feb0fe74e04b9) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:35.806122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef77951-cb74-4198-8519-26ecfe407fcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:35.807762 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef77951-cb74-4198-8519-26ecfe407fcb None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:35.814193 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 347/709] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:14:35 2022] GET /identity/v3/role_assignments?user.id=1da355c8be2d496bb08feb0fe74e04b9&scope.project.id=d18a5117a03b4f21ab3290a15680de66&role.id=f886b8e8889549e896a8fb3426cce9e1 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:35.815833 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee3d09-945e-4fb9-87e0-6efe1ac63cf2 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:35.816286 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee3d09-945e-4fb9-87e0-6efe1ac63cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:35.816418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f5e033-8ad1-45ad-aa37-df0222e53608 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:35.816870 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee3d09-945e-4fb9-87e0-6efe1ac63cf2 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:35.818623 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 348/710] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:35.824581 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa899e-c263-458c-8de7-0572e4627857 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:35.825104 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa899e-c263-458c-8de7-0572e4627857 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:35.825548 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa899e-c263-458c-8de7-0572e4627857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:35.848170 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-32fa899e-c263-458c-8de7-0572e4627857 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:35.849676 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-32fa899e-c263-458c-8de7-0572e4627857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:35.874012 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091d751-5f26-4d5f-a953-55827e14198e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:35.874778 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091d751-5f26-4d5f-a953-55827e14198e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:35.875165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091d751-5f26-4d5f-a953-55827e14198e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:35.875543 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a091d751-5f26-4d5f-a953-55827e14198e None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:35.877209 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a091d751-5f26-4d5f-a953-55827e14198e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:35.877643 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a091d751-5f26-4d5f-a953-55827e14198e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:35.879530 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a091d751-5f26-4d5f-a953-55827e14198e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:35.898863 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 363/711] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:14:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:35.907343 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091d751-5f26-4d5f-a953-55827e14198e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:35.915146 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:35.938878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 349/712] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:35.944156 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa899e-c263-458c-8de7-0572e4627857 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:35.953856 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:36.045791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a091d751-5f26-4d5f-a953-55827e14198e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:36.097311 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa899e-c263-458c-8de7-0572e4627857 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:36.103702 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8dfe3-d1f7-49a4-be65-d11feec7917f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:36.104351 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8dfe3-d1f7-49a4-be65-d11feec7917f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:36.104633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8dfe3-d1f7-49a4-be65-d11feec7917f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:36.104911 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a8dfe3-d1f7-49a4-be65-d11feec7917f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:36.105656 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8dfe3-d1f7-49a4-be65-d11feec7917f None admin] RBAC: Adding query filter params (role.id=faf1138f11d34abe8a862f1a1f7c140a, scope.project.id=d18a5117a03b4f21ab3290a15680de66, user.id=30fe233a7d4c4fbea6ede233bd9ff8c8) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:36.106433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8dfe3-d1f7-49a4-be65-d11feec7917f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:36.112817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a8dfe3-d1f7-49a4-be65-d11feec7917f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:36.122507 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 364/713] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:14:35 2022] GET /identity/v3/role_assignments?user.id=30fe233a7d4c4fbea6ede233bd9ff8c8&scope.project.id=d18a5117a03b4f21ab3290a15680de66&role.id=faf1138f11d34abe8a862f1a1f7c140a => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:36.148759 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebafcc-d9b4-4a78-bbd6-41a405ab8d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:36.149401 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebafcc-d9b4-4a78-bbd6-41a405ab8d25 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:36.149733 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebafcc-d9b4-4a78-bbd6-41a405ab8d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:36.150018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebafcc-d9b4-4a78-bbd6-41a405ab8d25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:36.162309 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ebafcc-d9b4-4a78-bbd6-41a405ab8d25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:36.163875 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ebafcc-d9b4-4a78-bbd6-41a405ab8d25 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:36.180763 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-22ebafcc-d9b4-4a78-bbd6-41a405ab8d25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:36.180763 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:36.180763 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:36.180763 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:36.180763 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:36.180763 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:36.180763 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:36.180763 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:36.180763 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:36.180763 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:36.180763 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:36.180763 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:36.180763 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:36.180763 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:36.180763 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:36.180763 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:36.181879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:36.181879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:36.181879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:36.181879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:36.181879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:36.181879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:36.181879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:36.181879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:36.181879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:36.181879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:36.181879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:36.181879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:36.181879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:36.181879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:36.181879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:36.181879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:36.181879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:36.182803 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:36.182803 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:36.182803 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:36.182803 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 350/714] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:36.188285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8dfe3-d1f7-49a4-be65-d11feec7917f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:36.297977 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebafcc-d9b4-4a78-bbd6-41a405ab8d25 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:36.313902 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:36.322085 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a8dfe3-d1f7-49a4-be65-d11feec7917f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:36.366872 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f5cfa-b9ba-42c9-9f2a-fda08d578769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:36.367362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f5cfa-b9ba-42c9-9f2a-fda08d578769 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:36.367644 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f5cfa-b9ba-42c9-9f2a-fda08d578769 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:36.367914 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f5cfa-b9ba-42c9-9f2a-fda08d578769 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:36.368564 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967f5cfa-b9ba-42c9-9f2a-fda08d578769 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:36.368872 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967f5cfa-b9ba-42c9-9f2a-fda08d578769 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:36.370232 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967f5cfa-b9ba-42c9-9f2a-fda08d578769 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:36.376962 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:36.391602 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 365/715] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:36.398047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f5cfa-b9ba-42c9-9f2a-fda08d578769 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:36.471520 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebafcc-d9b4-4a78-bbd6-41a405ab8d25 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:36.519109 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f5cfa-b9ba-42c9-9f2a-fda08d578769 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:36.520886 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2552144-f0e1-4be6-935e-b059842187ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:36.523355 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2552144-f0e1-4be6-935e-b059842187ca None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:36.523726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2552144-f0e1-4be6-935e-b059842187ca None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:36.524003 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2552144-f0e1-4be6-935e-b059842187ca None admin] PATH_INFO: `/v3/services/network` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:36.524614 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2552144-f0e1-4be6-935e-b059842187ca None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:36.525995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2552144-f0e1-4be6-935e-b059842187ca None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:36.546380 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-f2552144-f0e1-4be6-935e-b059842187ca None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 02 09:14:36.546380 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:36.546380 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 09:14:36.546380 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 09:14:36.546380 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 09:14:36.546380 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 09:14:36.546380 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 09:14:36.546380 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:14:36.546380 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 02 09:14:36.546380 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:36.546380 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:14:36.546380 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:36.546380 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:36.546380 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:36.546380 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:36.546380 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:36.546380 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:36.547531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:36.547531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:36.547531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:36.547531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:36.547531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:36.547531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:36.547531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 09:14:36.547531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 09:14:36.547531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 09:14:36.547531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 09:14:36.547531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:36.547531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:36.547531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:36.547531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:36.547531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:36.547531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:36.548369 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:36.548369 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:36.548369 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:36.548369 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:36.548369 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:36.548369 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:36.548369 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:36.548369 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:36.548369 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:36.548369 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:36.548369 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 09:14:36.548369 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:14:36.548369 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 02 09:14:36.548369 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:36.551719 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 351/716] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:36 2022] GET /identity/v3/services/network => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:14:36.557486 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2552144-f0e1-4be6-935e-b059842187ca None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:36.572229 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b054c-1b09-42a9-be84-cf25d6f70366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:36.573137 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b054c-1b09-42a9-be84-cf25d6f70366 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:36.573421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b054c-1b09-42a9-be84-cf25d6f70366 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:36.573692 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b054c-1b09-42a9-be84-cf25d6f70366 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:36.574343 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6b054c-1b09-42a9-be84-cf25d6f70366 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:36.575963 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6b054c-1b09-42a9-be84-cf25d6f70366 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:36.666803 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:36.670677 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2552144-f0e1-4be6-935e-b059842187ca None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:36.689730 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 366/717] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Sep 2 09:14:36 2022] POST /identity/v3/users => generated 254 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:14:36.705977 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80040eb-9041-4284-a6da-426c7a4b4dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:36.706601 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80040eb-9041-4284-a6da-426c7a4b4dd4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:36.706907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80040eb-9041-4284-a6da-426c7a4b4dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:36.707179 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80040eb-9041-4284-a6da-426c7a4b4dd4 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:36.707817 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80040eb-9041-4284-a6da-426c7a4b4dd4 None admin] RBAC: Adding query filter params (name=network) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:36.708120 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80040eb-9041-4284-a6da-426c7a4b4dd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:36.710035 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80040eb-9041-4284-a6da-426c7a4b4dd4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:36.715300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 352/718] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Sep 2 09:14:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:36.720725 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b054c-1b09-42a9-be84-cf25d6f70366 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:36.740085 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:36.865176 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b054c-1b09-42a9-be84-cf25d6f70366 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:36.902023 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d199d3f-3b7a-4b2a-9ef6-da19c2d04af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:36.902657 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d199d3f-3b7a-4b2a-9ef6-da19c2d04af2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:36.902978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d199d3f-3b7a-4b2a-9ef6-da19c2d04af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:36.903252 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d199d3f-3b7a-4b2a-9ef6-da19c2d04af2 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:36.903902 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d199d3f-3b7a-4b2a-9ef6-da19c2d04af2 None admin] RBAC: Adding query filter params (type=network) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:36.904213 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d199d3f-3b7a-4b2a-9ef6-da19c2d04af2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:36.905601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d199d3f-3b7a-4b2a-9ef6-da19c2d04af2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:36.914871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 367/719] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Sep 2 09:14:36 2022] GET /identity/v3/services?type=network => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:38.047983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f166315-3a03-4954-993a-491c687ed1b7 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:38.048496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f166315-3a03-4954-993a-491c687ed1b7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:38.048496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f166315-3a03-4954-993a-491c687ed1b7 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:38.050112 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 353/720] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:38.055546 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69906bf8-27fb-4b7a-8ef9-4873338bca77 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:38.055973 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69906bf8-27fb-4b7a-8ef9-4873338bca77 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:38.056069 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69906bf8-27fb-4b7a-8ef9-4873338bca77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:38.074673 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-69906bf8-27fb-4b7a-8ef9-4873338bca77 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:38.075888 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-69906bf8-27fb-4b7a-8ef9-4873338bca77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:38.080683 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7641f-3df4-4471-a924-1db1f46f780e None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:38.080966 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7641f-3df4-4471-a924-1db1f46f780e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:38.081241 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7641f-3df4-4471-a924-1db1f46f780e None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:38.082849 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 354/721] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:14:38.091864 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72a5f2-b765-406b-9153-9ad82e75bb8c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:38.092142 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72a5f2-b765-406b-9153-9ad82e75bb8c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:38.092445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f72a5f2-b765-406b-9153-9ad82e75bb8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:38.123816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0f72a5f2-b765-406b-9153-9ad82e75bb8c None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:38.124946 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0f72a5f2-b765-406b-9153-9ad82e75bb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:38.160124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 368/722] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:38.237026 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 355/723] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:38.524018 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69906bf8-27fb-4b7a-8ef9-4873338bca77 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:38.527082 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:38.628396 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69906bf8-27fb-4b7a-8ef9-4873338bca77 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:38.647970 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88621a1-b4e0-4c75-ac33-c1fa810ac6d9 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:38.648281 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88621a1-b4e0-4c75-ac33-c1fa810ac6d9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:38.648522 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88621a1-b4e0-4c75-ac33-c1fa810ac6d9 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:38.650153 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 356/724] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:38.655411 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21ac38-9b7b-4f6a-9d2c-e644899f2c1f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:38.655703 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21ac38-9b7b-4f6a-9d2c-e644899f2c1f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:38.655981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21ac38-9b7b-4f6a-9d2c-e644899f2c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:38.675495 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d40a62-159b-4a68-a40f-0f56603a2f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:38.681908 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d40a62-159b-4a68-a40f-0f56603a2f23 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:38.681908 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d40a62-159b-4a68-a40f-0f56603a2f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:38.681908 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d40a62-159b-4a68-a40f-0f56603a2f23 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:38.681908 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d40a62-159b-4a68-a40f-0f56603a2f23 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:38.681908 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d40a62-159b-4a68-a40f-0f56603a2f23 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:38.693011 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ad21ac38-9b7b-4f6a-9d2c-e644899f2c1f None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:38.694904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ad21ac38-9b7b-4f6a-9d2c-e644899f2c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:38.766659 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 369/725] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Sep 2 09:14:38 2022] POST /identity/v3/services => generated 251 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:14:38.767592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d40a62-159b-4a68-a40f-0f56603a2f23 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:38.792094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 357/726] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:38 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:38.859349 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:38.904439 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d40a62-159b-4a68-a40f-0f56603a2f23 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:38.941660 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad39fe9-fbe6-447d-99ff-73a9b3644a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:38.942313 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad39fe9-fbe6-447d-99ff-73a9b3644a4c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:38.942665 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad39fe9-fbe6-447d-99ff-73a9b3644a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:38.942988 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad39fe9-fbe6-447d-99ff-73a9b3644a4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:38.965737 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad39fe9-fbe6-447d-99ff-73a9b3644a4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:38.971071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-cad39fe9-fbe6-447d-99ff-73a9b3644a4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:14:38.971071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:38.971071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:38.971071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:38.971071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:38.971071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:38.971071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:38.971071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:38.971071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:38.971071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:38.971071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:38.971071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:38.971071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:38.971071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:38.971071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:38.971071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:38.972011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:38.972011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:38.972011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:38.972011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:38.972011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:38.972011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:38.972011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:38.972011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:38.972011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:38.972011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:38.972011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:38.972011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:38.972011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:38.972011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:38.972011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:38.972011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:38.973092 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:38.973092 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:38.973092 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:38.973092 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:38.973092 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:38.973092 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:38.973092 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:38.973092 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:38.973092 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:38.973092 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:38.973092 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:14:38.973092 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:38.973092 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 370/727] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:14:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:38.980639 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21ac38-9b7b-4f6a-9d2c-e644899f2c1f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:39.004952 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:39.095736 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21ac38-9b7b-4f6a-9d2c-e644899f2c1f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:39.133495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63358e25-4cbb-44b9-9eea-cbb10b30360e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:39.134534 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63358e25-4cbb-44b9-9eea-cbb10b30360e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:39.135080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63358e25-4cbb-44b9-9eea-cbb10b30360e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:39.135569 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63358e25-4cbb-44b9-9eea-cbb10b30360e None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:39.136453 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63358e25-4cbb-44b9-9eea-cbb10b30360e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:39.136988 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63358e25-4cbb-44b9-9eea-cbb10b30360e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:39.138568 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63358e25-4cbb-44b9-9eea-cbb10b30360e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:39.148008 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 358/728] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:14:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:39.157740 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad39fe9-fbe6-447d-99ff-73a9b3644a4c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:39.163529 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935f2d3-ec7f-45c1-a8a6-1c4e15b9cf14 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:39.164405 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935f2d3-ec7f-45c1-a8a6-1c4e15b9cf14 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:39.166137 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935f2d3-ec7f-45c1-a8a6-1c4e15b9cf14 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:39.168070 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 359/729] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:39.168820 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935f2d3-ec7f-45c1-a8a6-1c4e15b9cf14 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:39.274257 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad39fe9-fbe6-447d-99ff-73a9b3644a4c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:39.287185 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:39.302995 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:39.308970 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935f2d3-ec7f-45c1-a8a6-1c4e15b9cf14 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:39.314981 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab94f8d-04af-471b-82a1-50b6820a9d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:39.315628 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab94f8d-04af-471b-82a1-50b6820a9d31 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:39.315911 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab94f8d-04af-471b-82a1-50b6820a9d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:39.316187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab94f8d-04af-471b-82a1-50b6820a9d31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:39.328020 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab94f8d-04af-471b-82a1-50b6820a9d31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:39.329509 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab94f8d-04af-471b-82a1-50b6820a9d31 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:39.344506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-6ab94f8d-04af-471b-82a1-50b6820a9d31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:39.344506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:39.344506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:39.344506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:39.344506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:39.344506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:39.344506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:39.344506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:39.344506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:39.344506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:39.344506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:39.344506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:39.344506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:39.344506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:39.344506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:39.344506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:39.345510 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:39.345510 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:39.345510 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:39.345510 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:39.345510 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:39.345510 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:39.345510 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:39.345510 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:39.345510 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:39.345510 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:39.345510 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:39.345510 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:39.345510 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:39.345510 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:39.345510 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:39.345510 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:39.345510 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:39.346450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:39.346450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:39.346450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:39.346450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 371/730] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:39.346688 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213f0ca-0b52-4719-88e6-36300ee43f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:39.347296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213f0ca-0b52-4719-88e6-36300ee43f8b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:39.347490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213f0ca-0b52-4719-88e6-36300ee43f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:39.347639 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9213f0ca-0b52-4719-88e6-36300ee43f8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:39.350948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa964c5d-0537-4d8e-8002-d3cf95924b15 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:39.350948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa964c5d-0537-4d8e-8002-d3cf95924b15 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:39.350948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa964c5d-0537-4d8e-8002-d3cf95924b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:39.360890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9213f0ca-0b52-4719-88e6-36300ee43f8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:39.366335 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-9213f0ca-0b52-4719-88e6-36300ee43f8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:14:39.366335 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:39.366335 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:39.366335 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:39.366335 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:39.366335 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:39.366335 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:39.366335 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:39.366335 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:39.366335 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:39.366335 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:39.366335 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:39.366335 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:39.366335 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:39.366335 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:39.366335 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:39.368378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:39.368378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:39.368378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:39.368378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:39.368378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:39.368378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:39.368378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:39.368378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:39.368378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:39.368378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:39.368378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:39.368378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:39.368378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:39.368378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:39.368378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:39.368378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:39.369556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:39.369556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:39.369556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:39.369556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:39.369556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:39.369556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:39.369556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:39.369556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:39.369556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:39.369556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:39.369556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:14:39.369556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:39.374728 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 360/731] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:14:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:39.374728 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213f0ca-0b52-4719-88e6-36300ee43f8b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:39.374931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-aa964c5d-0537-4d8e-8002-d3cf95924b15 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:39.374931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-aa964c5d-0537-4d8e-8002-d3cf95924b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:39.447355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 372/732] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:39 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:39.448715 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa964c5d-0537-4d8e-8002-d3cf95924b15 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:39.503241 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9213f0ca-0b52-4719-88e6-36300ee43f8b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:39.554171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa964c5d-0537-4d8e-8002-d3cf95924b15 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:39.562326 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:39.566428 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81213ce2-9d7d-426b-a85e-036ae75b8cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:39.567076 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81213ce2-9d7d-426b-a85e-036ae75b8cdd None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:39.567360 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81213ce2-9d7d-426b-a85e-036ae75b8cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:39.567630 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81213ce2-9d7d-426b-a85e-036ae75b8cdd None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:39.568275 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81213ce2-9d7d-426b-a85e-036ae75b8cdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:39.568608 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81213ce2-9d7d-426b-a85e-036ae75b8cdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:39.569961 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81213ce2-9d7d-426b-a85e-036ae75b8cdd None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:39.592199 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3c302-e7a5-487c-b3dd-44a93c36ce2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:39.592899 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3c302-e7a5-487c-b3dd-44a93c36ce2a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:39.593183 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3c302-e7a5-487c-b3dd-44a93c36ce2a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:39.593459 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3c302-e7a5-487c-b3dd-44a93c36ce2a None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:39.593707 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:39.594143 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c3c302-e7a5-487c-b3dd-44a93c36ce2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:39.594452 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c3c302-e7a5-487c-b3dd-44a93c36ce2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:39.595956 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c3c302-e7a5-487c-b3dd-44a93c36ce2a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:39.605019 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 361/733] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:39.606454 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 373/734] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:14:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:39.610846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81213ce2-9d7d-426b-a85e-036ae75b8cdd None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:39.615714 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3c302-e7a5-487c-b3dd-44a93c36ce2a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:39.735902 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81213ce2-9d7d-426b-a85e-036ae75b8cdd None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:39.758507 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3c302-e7a5-487c-b3dd-44a93c36ce2a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:39.772863 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66ee0e-cf3e-4797-9712-f3c8ab4665a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:39.773488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66ee0e-cf3e-4797-9712-f3c8ab4665a8 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:39.773765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66ee0e-cf3e-4797-9712-f3c8ab4665a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:39.774036 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66ee0e-cf3e-4797-9712-f3c8ab4665a8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:39.781588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66ee0e-cf3e-4797-9712-f3c8ab4665a8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:39.783038 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66ee0e-cf3e-4797-9712-f3c8ab4665a8 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:39.795009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-1b66ee0e-cf3e-4797-9712-f3c8ab4665a8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 02 09:14:39.795009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:39.795009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:39.795009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:39.795009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:39.795009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:39.795009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:39.795009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:39.795009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:39.795009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:39.795009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:39.795009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:39.795009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:14:39.795009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:14:39.795009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:14:39.795009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:14:39.796180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:39.796180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:39.796180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:14:39.796180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:39.796180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:14:39.796180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:39.796180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:39.796180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:39.796180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:39.796180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:39.796180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:39.796180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:39.796180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:39.796180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:39.796180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:39.796180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:39.796180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:39.796180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:39.797370 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:39.797370 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:39.797370 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:14:39.797370 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:14:39.797370 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:14:39.797370 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:14:39.797370 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:14:39.797370 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:14:39.797370 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 02 09:14:39.797370 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:39.800725 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3886f2-6353-4e4e-8470-f7e12a6e8317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:39.800867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 362/735] 10.222.0.57 () {56 vars in 1093 bytes} [Fri Sep 2 09:14:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:39.801495 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3886f2-6353-4e4e-8470-f7e12a6e8317 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:39.801846 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3886f2-6353-4e4e-8470-f7e12a6e8317 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:39.802085 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9467f-0fe3-49fa-b934-b03ebfa437c7 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:39.802246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3886f2-6353-4e4e-8470-f7e12a6e8317 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:39.802371 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9467f-0fe3-49fa-b934-b03ebfa437c7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:39.802693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9467f-0fe3-49fa-b934-b03ebfa437c7 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:39.804305 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 363/736] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:39.805004 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9467f-0fe3-49fa-b934-b03ebfa437c7 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:39.815518 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3886f2-6353-4e4e-8470-f7e12a6e8317 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:39.817552 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3886f2-6353-4e4e-8470-f7e12a6e8317 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:39.838436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-0a3886f2-6353-4e4e-8470-f7e12a6e8317 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:39.838436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:39.838436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:39.838436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:39.838436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:39.838436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:39.838436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:39.838436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:39.838436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:39.838436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:39.838436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:39.838436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:39.838436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:39.838436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:39.838436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:39.838436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:39.839608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:39.839608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:39.839608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:39.839608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:39.839608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:39.839608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:39.839608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:39.839608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:39.839608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:39.839608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:39.839608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:39.839608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:39.839608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:39.839608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:39.839608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:39.839608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:39.839608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:39.840819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:39.840819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:39.840819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:39.844522 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 374/737] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:39.845800 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3e4c6-52ab-473e-8833-1c6ae2ff6448 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:39.846093 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3e4c6-52ab-473e-8833-1c6ae2ff6448 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:39.846550 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a3e4c6-52ab-473e-8833-1c6ae2ff6448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:39.865009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-10a3e4c6-52ab-473e-8833-1c6ae2ff6448 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:39.866169 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-10a3e4c6-52ab-473e-8833-1c6ae2ff6448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:39.876785 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:39.881797 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:39.920104 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9467f-0fe3-49fa-b934-b03ebfa437c7 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:39.941405 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 375/738] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:39 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 02 09:14:39.942309 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a3e4c6-52ab-473e-8833-1c6ae2ff6448 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:39.960973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfcb0cb-c126-44c6-ac84-18cdfe9646e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:39.961864 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcb0cb-c126-44c6-ac84-18cdfe9646e4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:39.962387 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcb0cb-c126-44c6-ac84-18cdfe9646e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:39.962895 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcb0cb-c126-44c6-ac84-18cdfe9646e4 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:39.963759 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfcb0cb-c126-44c6-ac84-18cdfe9646e4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:39.964048 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfcb0cb-c126-44c6-ac84-18cdfe9646e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:39.965527 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfcb0cb-c126-44c6-ac84-18cdfe9646e4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:39.989243 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 364/739] 10.222.0.57 () {56 vars in 1133 bytes} [Fri Sep 2 09:14:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:39.989243 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfcb0cb-c126-44c6-ac84-18cdfe9646e4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:40.082916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a3e4c6-52ab-473e-8833-1c6ae2ff6448 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:40.106914 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfcb0cb-c126-44c6-ac84-18cdfe9646e4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:40.121388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815da488-4936-4e11-824b-470480f4c2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:40.121930 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815da488-4936-4e11-824b-470480f4c2bc None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:40.122208 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815da488-4936-4e11-824b-470480f4c2bc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:40.122477 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815da488-4936-4e11-824b-470480f4c2bc None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:40.123494 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815da488-4936-4e11-824b-470480f4c2bc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:40.125123 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815da488-4936-4e11-824b-470480f4c2bc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:40.131624 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:40.149340 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aad68c-135c-4509-a501-ac55ff1a076a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:40.149968 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aad68c-135c-4509-a501-ac55ff1a076a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:40.150252 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aad68c-135c-4509-a501-ac55ff1a076a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:40.150527 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aad68c-135c-4509-a501-ac55ff1a076a None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:40.151242 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aad68c-135c-4509-a501-ac55ff1a076a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:40.151552 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aad68c-135c-4509-a501-ac55ff1a076a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:40.152990 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aad68c-135c-4509-a501-ac55ff1a076a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:40.166163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 365/740] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:40.167021 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aad68c-135c-4509-a501-ac55ff1a076a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:40.222919 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 376/741] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Sep 2 09:14:39 2022] POST /identity/v3/services => generated 240 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:14:40.223849 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815da488-4936-4e11-824b-470480f4c2bc None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:40.242605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:40.297640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aad68c-135c-4509-a501-ac55ff1a076a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:40.354436 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a7214-e6a2-446f-a888-1cae95d6cc65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:40.355605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a7214-e6a2-446f-a888-1cae95d6cc65 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:40.356184 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a7214-e6a2-446f-a888-1cae95d6cc65 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:40.356754 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a7214-e6a2-446f-a888-1cae95d6cc65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:40.372461 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a7214-e6a2-446f-a888-1cae95d6cc65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:40.373851 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8a7214-e6a2-446f-a888-1cae95d6cc65 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:40.377529 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815da488-4936-4e11-824b-470480f4c2bc None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:40.392710 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-8b8a7214-e6a2-446f-a888-1cae95d6cc65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:40.392710 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:40.392710 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:40.392710 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:40.392710 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:40.392710 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:40.392710 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:40.392710 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:40.392710 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:40.392710 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:40.392710 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:40.392710 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:40.392710 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:40.392710 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:40.392710 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:40.392710 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:40.393879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:40.393879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:40.393879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:40.393879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:40.393879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:40.393879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:40.393879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:40.393879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:40.393879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:40.393879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:40.393879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:40.393879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:40.393879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:40.393879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:40.393879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:40.393879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:40.393879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:40.394870 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:40.394870 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:40.394870 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:40.394870 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 366/742] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:40.394870 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a7214-e6a2-446f-a888-1cae95d6cc65 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:40.418152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb31a7-4c39-4179-ac56-5c1591ca017f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:40.418994 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb31a7-4c39-4179-ac56-5c1591ca017f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:40.419204 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb31a7-4c39-4179-ac56-5c1591ca017f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:40.419574 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb31a7-4c39-4179-ac56-5c1591ca017f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:40.428762 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:40.429601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb31a7-4c39-4179-ac56-5c1591ca017f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:40.431140 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb31a7-4c39-4179-ac56-5c1591ca017f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:40.443460 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-b2cb31a7-4c39-4179-ac56-5c1591ca017f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 02 09:14:40.443460 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:40.443460 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:40.443460 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:40.443460 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:40.443460 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:40.443460 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:40.443460 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:40.443460 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:40.443460 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:40.443460 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:40.443460 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:40.443460 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:14:40.443460 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:14:40.443460 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:14:40.443460 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:14:40.445102 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:40.445102 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:40.445102 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:14:40.445102 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:40.445102 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:14:40.445102 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:40.445102 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:40.445102 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:40.445102 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:40.445102 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:40.445102 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:40.445102 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:40.445102 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:40.445102 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:40.445102 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:40.445102 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:40.445102 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:40.445102 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:40.446029 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:40.446029 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:40.446029 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:14:40.446029 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:14:40.446029 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:14:40.446029 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:14:40.446029 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:14:40.446029 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:14:40.446029 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 02 09:14:40.446029 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:40.446029 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 377/743] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:14:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:40.446621 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb31a7-4c39-4179-ac56-5c1591ca017f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:40.522503 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:40.524329 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a7214-e6a2-446f-a888-1cae95d6cc65 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:40.554514 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb31a7-4c39-4179-ac56-5c1591ca017f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:40.559807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f1187-e561-48db-a7bc-d8c74b3924eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:40.560715 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f1187-e561-48db-a7bc-d8c74b3924eb None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:40.561192 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f1187-e561-48db-a7bc-d8c74b3924eb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:40.562592 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f1187-e561-48db-a7bc-d8c74b3924eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:40.575386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f1187-e561-48db-a7bc-d8c74b3924eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:40.581321 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-b10f1187-e561-48db-a7bc-d8c74b3924eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:40.581321 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:40.581321 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:40.581321 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:40.581321 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:40.581321 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:40.581321 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:40.581321 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:40.581321 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:40.581321 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:40.581321 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:40.581321 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:40.581321 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:40.581321 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:40.581321 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:40.581321 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:40.582254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:40.582254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:40.582254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:40.582254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:40.582254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:40.582254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:40.582254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:40.582254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:40.582254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:40.582254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:40.582254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:40.582254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:40.582254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:40.582254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:40.582254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:40.582254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:40.583138 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:40.583138 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:40.583138 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:40.583138 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:40.583138 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:40.583138 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:40.583138 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:40.583138 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:40.583138 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:40.583138 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:40.583138 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:40.583138 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:40.585848 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 367/744] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Sep 2 09:14:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:40.586984 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f1187-e561-48db-a7bc-d8c74b3924eb None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:40.591327 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11895ece-0321-47f9-b43d-b925be6764f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:40.591959 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11895ece-0321-47f9-b43d-b925be6764f6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:40.593023 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11895ece-0321-47f9-b43d-b925be6764f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:40.593023 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11895ece-0321-47f9-b43d-b925be6764f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:40.593378 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11895ece-0321-47f9-b43d-b925be6764f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:40.593713 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11895ece-0321-47f9-b43d-b925be6764f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:40.595140 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11895ece-0321-47f9-b43d-b925be6764f6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:40.613871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 378/745] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:40.613871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11895ece-0321-47f9-b43d-b925be6764f6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:40.687250 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:40.707969 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f1187-e561-48db-a7bc-d8c74b3924eb None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:40.717930 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11895ece-0321-47f9-b43d-b925be6764f6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:40.749735 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:40.756700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ccb1a8-daeb-46f5-9737-535ab48de1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:40.757357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ccb1a8-daeb-46f5-9737-535ab48de1e9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:40.757713 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ccb1a8-daeb-46f5-9737-535ab48de1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:40.758060 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ccb1a8-daeb-46f5-9737-535ab48de1e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:40.758850 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ccb1a8-daeb-46f5-9737-535ab48de1e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:40.759239 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954e1ffe-d06a-4295-b4e5-cbdc94dee88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:40.759370 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ccb1a8-daeb-46f5-9737-535ab48de1e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:40.760145 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e1ffe-d06a-4295-b4e5-cbdc94dee88d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:40.760620 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e1ffe-d06a-4295-b4e5-cbdc94dee88d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:40.760902 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ccb1a8-daeb-46f5-9737-535ab48de1e9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:40.761254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e1ffe-d06a-4295-b4e5-cbdc94dee88d None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:40.762320 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954e1ffe-d06a-4295-b4e5-cbdc94dee88d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:40.762827 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954e1ffe-d06a-4295-b4e5-cbdc94dee88d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:40.765187 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954e1ffe-d06a-4295-b4e5-cbdc94dee88d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:40.769777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 379/746] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Sep 2 09:14:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:40.770916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ccb1a8-daeb-46f5-9737-535ab48de1e9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:40.789885 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 368/747] 10.222.0.57 () {56 vars in 1135 bytes} [Fri Sep 2 09:14:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:40.791030 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954e1ffe-d06a-4295-b4e5-cbdc94dee88d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:40.908229 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ccb1a8-daeb-46f5-9737-535ab48de1e9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:40.914434 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954e1ffe-d06a-4295-b4e5-cbdc94dee88d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:40.952084 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6017608d-d6f5-4b54-86bf-318ff906e801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:40.952448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ffb57-b65b-47d9-82f9-884e9e6c3743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:40.952676 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017608d-d6f5-4b54-86bf-318ff906e801 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:40.952919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017608d-d6f5-4b54-86bf-318ff906e801 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:40.953197 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6017608d-d6f5-4b54-86bf-318ff906e801 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:40.954088 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ffb57-b65b-47d9-82f9-884e9e6c3743 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:40.954579 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ffb57-b65b-47d9-82f9-884e9e6c3743 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:40.955443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ffb57-b65b-47d9-82f9-884e9e6c3743 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:40.964421 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6017608d-d6f5-4b54-86bf-318ff906e801 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:40.964847 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8ffb57-b65b-47d9-82f9-884e9e6c3743 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:40.965867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6017608d-d6f5-4b54-86bf-318ff906e801 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:40.966944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8ffb57-b65b-47d9-82f9-884e9e6c3743 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:40.968554 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:40.987418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-dc8ffb57-b65b-47d9-82f9-884e9e6c3743 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:40.987418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:40.987418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:40.987418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:40.987418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:40.987418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:40.987418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:40.987418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:40.987418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:40.987418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:40.987418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:40.987418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:40.987418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:14:40.987418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:14:40.987418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:14:40.988932 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:14:40.988932 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:40.988932 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:40.988932 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:40.988932 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:40.988932 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:40.988932 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:40.988932 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:40.988932 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:40.988932 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:40.988932 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:40.988932 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:40.988932 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:40.988932 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:40.988932 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:40.988932 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:40.988932 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:40.989821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:14:40.989821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:14:40.989821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:14:40.989821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:14:40.989821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:14:40.989821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:14:40.989821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:40.989821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:40.991574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-6017608d-d6f5-4b54-86bf-318ff906e801 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:40.991574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:40.991574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:40.991574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:40.991574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:40.991574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:40.991574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:40.991574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:40.991574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:40.991574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:40.991574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:40.991574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:40.991574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:40.991574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:40.991574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:40.991574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:40.992473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:40.992473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:40.992473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:40.992473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:40.992473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:40.992473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:40.992473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:40.992473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:40.992473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:40.992473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:40.992473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:40.992473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:40.992473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:40.992473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:40.992473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:40.992473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:40.992473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:40.993364 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:40.993364 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:40.993364 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:40.993859 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 380/748] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:40.994737 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ffb57-b65b-47d9-82f9-884e9e6c3743 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:40.995352 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 369/749] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:40.996285 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6017608d-d6f5-4b54-86bf-318ff906e801 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:41.056498 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:41.105539 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ffb57-b65b-47d9-82f9-884e9e6c3743 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:41.107762 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6017608d-d6f5-4b54-86bf-318ff906e801 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:41.141536 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf269ab-ffef-4de6-86f8-4f2a4cc8fba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:41.142143 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf269ab-ffef-4de6-86f8-4f2a4cc8fba5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:41.142423 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf269ab-ffef-4de6-86f8-4f2a4cc8fba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:41.142694 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf269ab-ffef-4de6-86f8-4f2a4cc8fba5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:41.144314 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19b624-a379-4b1b-ad57-a4a95c763f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:41.145054 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19b624-a379-4b1b-ad57-a4a95c763f25 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:41.145408 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19b624-a379-4b1b-ad57-a4a95c763f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:41.145752 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19b624-a379-4b1b-ad57-a4a95c763f25 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:41.146551 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19b624-a379-4b1b-ad57-a4a95c763f25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:41.146964 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19b624-a379-4b1b-ad57-a4a95c763f25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:41.148413 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19b624-a379-4b1b-ad57-a4a95c763f25 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:41.166239 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 370/750] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:14:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:41.166239 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19b624-a379-4b1b-ad57-a4a95c763f25 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:41.220374 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf269ab-ffef-4de6-86f8-4f2a4cc8fba5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:41.221839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf269ab-ffef-4de6-86f8-4f2a4cc8fba5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:41.228199 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:41.276733 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:41.277558 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19b624-a379-4b1b-ad57-a4a95c763f25 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:41.312237 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93ce59-8285-431a-a945-ef8244f75b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:41.312846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93ce59-8285-431a-a945-ef8244f75b0d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:41.313126 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93ce59-8285-431a-a945-ef8244f75b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:41.313391 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93ce59-8285-431a-a945-ef8244f75b0d None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:41.314031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93ce59-8285-431a-a945-ef8244f75b0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:41.314328 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93ce59-8285-431a-a945-ef8244f75b0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:41.315697 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93ce59-8285-431a-a945-ef8244f75b0d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:41.328580 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 371/751] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:41.329817 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8edc90-e431-4242-8069-fedc7e04dc81 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:41.330099 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8edc90-e431-4242-8069-fedc7e04dc81 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:41.330369 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8edc90-e431-4242-8069-fedc7e04dc81 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:41.335801 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 372/752] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:41.335801 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8edc90-e431-4242-8069-fedc7e04dc81 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:41.341064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 381/753] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:14:40 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 09:14:41.341064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf269ab-ffef-4de6-86f8-4f2a4cc8fba5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:41.444992 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8edc90-e431-4242-8069-fedc7e04dc81 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:41.452886 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf269ab-ffef-4de6-86f8-4f2a4cc8fba5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:41.460975 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:41.485222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb8c9b-4a51-4b29-8911-bd06fd5b099d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:41.485816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb8c9b-4a51-4b29-8911-bd06fd5b099d None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:41.486104 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb8c9b-4a51-4b29-8911-bd06fd5b099d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:41.486372 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb8c9b-4a51-4b29-8911-bd06fd5b099d None admin] PATH_INFO: `/v3/projects/d18a5117a03b4f21ab3290a15680de66/users/1da355c8be2d496bb08feb0fe74e04b9/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:41.488302 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deb8c9b-4a51-4b29-8911-bd06fd5b099d None admin] RBAC: Authorizing `identity:create_grant(project_id=d18a5117a03b4f21ab3290a15680de66, user_id=1da355c8be2d496bb08feb0fe74e04b9, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:41.489733 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deb8c9b-4a51-4b29-8911-bd06fd5b099d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:41.489961 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2366397c-50f7-4421-80af-74fe293fced2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:41.491262 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366397c-50f7-4421-80af-74fe293fced2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:41.491547 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366397c-50f7-4421-80af-74fe293fced2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:41.491822 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2366397c-50f7-4421-80af-74fe293fced2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:41.499014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2366397c-50f7-4421-80af-74fe293fced2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:41.500345 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2366397c-50f7-4421-80af-74fe293fced2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:41.511925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-2366397c-50f7-4421-80af-74fe293fced2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 09:14:41.511925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:41.511925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:41.511925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:41.511925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:41.511925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:41.511925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:41.511925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:41.511925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:41.511925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:41.511925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:41.511925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:41.511925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:14:41.511925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:14:41.511925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:14:41.511925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:14:41.513024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:41.513024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:41.513024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:14:41.513024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:41.513024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:14:41.513024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:41.513024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:41.513024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:41.513024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:41.513024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:41.513024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:41.513024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:41.513024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:41.513024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:41.513024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:41.513024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:41.513024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:41.513024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:41.514014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:41.514014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:41.514014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:14:41.514014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:14:41.514014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:14:41.514014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:14:41.514014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:14:41.514014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:14:41.514014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 09:14:41.514014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:41.514014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 382/754] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Sep 2 09:14:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:41.514617 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeaca3b-4d67-4ac2-baed-d96600850ccb None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:41.515164 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeaca3b-4d67-4ac2-baed-d96600850ccb None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:41.515164 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeaca3b-4d67-4ac2-baed-d96600850ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:41.532596 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3eeaca3b-4d67-4ac2-baed-d96600850ccb None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:41.533799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3eeaca3b-4d67-4ac2-baed-d96600850ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:41.541484 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:41.599042 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 383/755] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:14:41.600415 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeaca3b-4d67-4ac2-baed-d96600850ccb None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:41.623799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 373/756] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Sep 2 09:14:41 2022] PUT /identity/v3/projects/d18a5117a03b4f21ab3290a15680de66/users/1da355c8be2d496bb08feb0fe74e04b9/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:41.625468 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb8c9b-4a51-4b29-8911-bd06fd5b099d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:41.800446 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeaca3b-4d67-4ac2-baed-d96600850ccb None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:41.814816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb8c9b-4a51-4b29-8911-bd06fd5b099d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:41.837918 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609bf78-b470-49dc-a26b-b6e344cfe174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:41.838929 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609bf78-b470-49dc-a26b-b6e344cfe174 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:41.839520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609bf78-b470-49dc-a26b-b6e344cfe174 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:41.839994 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609bf78-b470-49dc-a26b-b6e344cfe174 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:41.851835 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6609bf78-b470-49dc-a26b-b6e344cfe174 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:41.852271 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152e431-e826-4d2b-94a1-e6f022209fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:41.852888 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152e431-e826-4d2b-94a1-e6f022209fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:41.853171 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152e431-e826-4d2b-94a1-e6f022209fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:41.853449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152e431-e826-4d2b-94a1-e6f022209fb2 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:41.854238 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152e431-e826-4d2b-94a1-e6f022209fb2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:41.854555 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152e431-e826-4d2b-94a1-e6f022209fb2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:41.854883 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6609bf78-b470-49dc-a26b-b6e344cfe174 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:41.856071 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152e431-e826-4d2b-94a1-e6f022209fb2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:41.857909 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:41.872505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-6609bf78-b470-49dc-a26b-b6e344cfe174 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:41.872505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:41.872505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:41.872505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:41.872505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:41.872505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:41.872505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:41.872505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:41.872505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:41.872505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:41.872505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:41.872505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:41.872505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:41.872505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:41.872505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:41.872505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:41.873483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:41.873483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:41.873483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:41.873483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:41.873483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:41.873483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:41.873483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:41.873483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:41.873483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:41.873483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:41.873483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:41.873483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:41.873483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:41.873483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:41.873483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:41.873483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:41.873483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:41.874458 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:41.874458 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:41.874458 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:41.876530 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 384/758] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:41.878338 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 374/758] 10.222.0.57 () {56 vars in 1149 bytes} [Fri Sep 2 09:14:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:41.883756 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609bf78-b470-49dc-a26b-b6e344cfe174 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:41.885059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152e431-e826-4d2b-94a1-e6f022209fb2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:41.922738 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:42.002098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152e431-e826-4d2b-94a1-e6f022209fb2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:42.017586 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609bf78-b470-49dc-a26b-b6e344cfe174 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:42.048421 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038016e2-8f05-4c13-bf4e-c91fc8aaa7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:42.049038 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038016e2-8f05-4c13-bf4e-c91fc8aaa7b9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:42.049319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038016e2-8f05-4c13-bf4e-c91fc8aaa7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:42.049591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038016e2-8f05-4c13-bf4e-c91fc8aaa7b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:42.057998 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2db9d-5729-4736-81e8-4fd9cc16ee44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:42.058547 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2db9d-5729-4736-81e8-4fd9cc16ee44 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:42.058861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2db9d-5729-4736-81e8-4fd9cc16ee44 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:42.059140 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e2db9d-5729-4736-81e8-4fd9cc16ee44 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:42.059799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2db9d-5729-4736-81e8-4fd9cc16ee44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:42.060158 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2db9d-5729-4736-81e8-4fd9cc16ee44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:42.061549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e2db9d-5729-4736-81e8-4fd9cc16ee44 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:42.081144 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 385/759] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:42.082055 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2db9d-5729-4736-81e8-4fd9cc16ee44 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:42.144012 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038016e2-8f05-4c13-bf4e-c91fc8aaa7b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:42.145542 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038016e2-8f05-4c13-bf4e-c91fc8aaa7b9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:42.147837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:42.180911 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:42.208446 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e2db9d-5729-4736-81e8-4fd9cc16ee44 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:42.253494 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758a8d4a-d64f-4bfc-898f-35f1cb772272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:42.254439 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a8d4a-d64f-4bfc-898f-35f1cb772272 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:42.254582 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 375/760] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:14:41 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 09:14:42.255057 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a8d4a-d64f-4bfc-898f-35f1cb772272 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:42.255499 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a8d4a-d64f-4bfc-898f-35f1cb772272 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:42.255670 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038016e2-8f05-4c13-bf4e-c91fc8aaa7b9 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:42.256951 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758a8d4a-d64f-4bfc-898f-35f1cb772272 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:42.258724 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758a8d4a-d64f-4bfc-898f-35f1cb772272 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:42.270302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-758a8d4a-d64f-4bfc-898f-35f1cb772272 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 02 09:14:42.270302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:42.270302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 09:14:42.270302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 09:14:42.270302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 09:14:42.270302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 09:14:42.270302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 09:14:42.270302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:14:42.270302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 02 09:14:42.270302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:42.270302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:14:42.270302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:42.270302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:42.270302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:42.270302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:42.270302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:42.270302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:42.271406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:42.271406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:42.271406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:42.271406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:42.271406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:42.271406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:42.271406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 09:14:42.271406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 09:14:42.271406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 09:14:42.271406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 09:14:42.271406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:42.271406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:42.271406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:42.271406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:42.271406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:42.271406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:42.273227 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:42.273227 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:42.273227 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:42.273227 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:42.273227 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:42.273227 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:42.273227 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:42.273227 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:42.273227 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:42.273227 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:42.273227 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 09:14:42.273227 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:14:42.273227 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 02 09:14:42.273227 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:42.275903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 386/761] 10.222.0.57 () {56 vars in 1129 bytes} [Fri Sep 2 09:14:42 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:42.277411 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9edf8-ef98-4e1b-8db8-2136e4602a91 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:42.278420 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9edf8-ef98-4e1b-8db8-2136e4602a91 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:42.278922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9edf8-ef98-4e1b-8db8-2136e4602a91 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:42.280666 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 387/762] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:42.281585 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be9edf8-ef98-4e1b-8db8-2136e4602a91 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:42.368679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038016e2-8f05-4c13-bf4e-c91fc8aaa7b9 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:42.393007 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be9edf8-ef98-4e1b-8db8-2136e4602a91 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:42.405118 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f074a0-23bf-4e21-a26f-848b0a98ebff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:42.405746 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f074a0-23bf-4e21-a26f-848b0a98ebff None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:42.406027 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f074a0-23bf-4e21-a26f-848b0a98ebff None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:42.406303 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f074a0-23bf-4e21-a26f-848b0a98ebff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:42.409456 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:42.413891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:42.415572 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f074a0-23bf-4e21-a26f-848b0a98ebff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:42.417142 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f074a0-23bf-4e21-a26f-848b0a98ebff None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:42.429090 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d6f074a0-23bf-4e21-a26f-848b0a98ebff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:42.429090 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:42.429090 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:42.429090 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:42.429090 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:42.429090 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:42.429090 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:42.429090 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:42.429090 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:42.429090 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:42.429090 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:42.429090 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:42.429090 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:14:42.429090 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:14:42.429090 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:14:42.429979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:14:42.429979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:42.429979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:42.429979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:42.429979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:42.429979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:42.429979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:42.429979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:42.429979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:42.429979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:42.429979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:42.429979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:42.429979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:42.429979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:42.429979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:42.429979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:42.429979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:42.431402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:14:42.431402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:14:42.431402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:14:42.431402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:14:42.431402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:14:42.431402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:14:42.431402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:42.431402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:42.431402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 376/763] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:42.431896 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e0ff3-db9d-4682-9b02-6bec178ddd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:42.431896 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e0ff3-db9d-4682-9b02-6bec178ddd8a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:42.431896 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e0ff3-db9d-4682-9b02-6bec178ddd8a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:42.432095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e0ff3-db9d-4682-9b02-6bec178ddd8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:42.433231 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f074a0-23bf-4e21-a26f-848b0a98ebff None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:42.446122 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e0ff3-db9d-4682-9b02-6bec178ddd8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:42.448105 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e0ff3-db9d-4682-9b02-6bec178ddd8a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:42.463416 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-fc3e0ff3-db9d-4682-9b02-6bec178ddd8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:42.463416 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:42.463416 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:42.463416 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:42.463416 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:42.463416 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:42.463416 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:42.463416 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:42.463416 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:42.463416 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:42.463416 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:42.463416 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:42.463416 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:42.463416 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:42.463416 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:42.463416 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:42.464361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:42.464361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:42.464361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:42.464361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:42.464361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:42.464361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:42.464361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:42.464361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:42.464361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:42.464361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:42.464361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:42.464361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:42.464361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:42.464361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:42.464361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:42.464361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:42.464361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:42.465235 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:42.465235 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:42.465235 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:42.466941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 388/764] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:42.468266 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f71db-9731-452a-9080-d70bbbe0687e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:42.468746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f71db-9731-452a-9080-d70bbbe0687e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:42.469162 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f71db-9731-452a-9080-d70bbbe0687e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:42.487734 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f30f71db-9731-452a-9080-d70bbbe0687e None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:42.489132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f30f71db-9731-452a-9080-d70bbbe0687e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:42.544712 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f074a0-23bf-4e21-a26f-848b0a98ebff None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:42.566740 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 389/765] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:14:42.566740 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f71db-9731-452a-9080-d70bbbe0687e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:42.592670 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa348a9-4c89-4de7-9d87-d2d9dfb89746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:42.593295 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa348a9-4c89-4de7-9d87-d2d9dfb89746 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:42.593630 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa348a9-4c89-4de7-9d87-d2d9dfb89746 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:42.593909 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa348a9-4c89-4de7-9d87-d2d9dfb89746 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:42.594849 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa348a9-4c89-4de7-9d87-d2d9dfb89746 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:42.595159 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa348a9-4c89-4de7-9d87-d2d9dfb89746 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:42.597330 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa348a9-4c89-4de7-9d87-d2d9dfb89746 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:42.603897 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 377/766] 10.222.0.57 () {56 vars in 1124 bytes} [Fri Sep 2 09:14:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:42.605269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa348a9-4c89-4de7-9d87-d2d9dfb89746 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:42.684308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:42.692306 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:42.696201 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f71db-9731-452a-9080-d70bbbe0687e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:42.729723 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa348a9-4c89-4de7-9d87-d2d9dfb89746 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:42.733749 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bcd472-e23f-46c6-b28d-779f4b021fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:42.734370 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bcd472-e23f-46c6-b28d-779f4b021fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:42.734655 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bcd472-e23f-46c6-b28d-779f4b021fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:42.734963 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bcd472-e23f-46c6-b28d-779f4b021fa4 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:42.735666 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bcd472-e23f-46c6-b28d-779f4b021fa4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:42.736018 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bcd472-e23f-46c6-b28d-779f4b021fa4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:42.738854 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bcd472-e23f-46c6-b28d-779f4b021fa4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:42.758478 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 390/767] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:14:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:42.758478 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bcd472-e23f-46c6-b28d-779f4b021fa4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:42.773213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e256ce4-4f35-46d7-9dba-ae25f7e72d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:42.774045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e256ce4-4f35-46d7-9dba-ae25f7e72d6c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:42.774477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e256ce4-4f35-46d7-9dba-ae25f7e72d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:42.774927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e256ce4-4f35-46d7-9dba-ae25f7e72d6c None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:42.775735 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e256ce4-4f35-46d7-9dba-ae25f7e72d6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:42.776203 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e256ce4-4f35-46d7-9dba-ae25f7e72d6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:42.777713 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e256ce4-4f35-46d7-9dba-ae25f7e72d6c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:42.795032 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 378/768] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:42.796150 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e256ce4-4f35-46d7-9dba-ae25f7e72d6c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:42.872091 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bcd472-e23f-46c6-b28d-779f4b021fa4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:42.906967 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e437d7-69c3-41e5-a83b-de7dd6d60981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:42.907583 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e437d7-69c3-41e5-a83b-de7dd6d60981 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:42.907860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e437d7-69c3-41e5-a83b-de7dd6d60981 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:42.908132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e437d7-69c3-41e5-a83b-de7dd6d60981 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:42.908758 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e437d7-69c3-41e5-a83b-de7dd6d60981 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:42.909062 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e437d7-69c3-41e5-a83b-de7dd6d60981 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:42.910388 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e256ce4-4f35-46d7-9dba-ae25f7e72d6c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:42.910493 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e437d7-69c3-41e5-a83b-de7dd6d60981 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:42.916716 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 391/769] 10.222.0.57 () {56 vars in 1124 bytes} [Fri Sep 2 09:14:42 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:42.919590 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e437d7-69c3-41e5-a83b-de7dd6d60981 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:42.931030 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:42.949555 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddead1-c768-48b7-ac90-e788472106b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:42.950105 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddead1-c768-48b7-ac90-e788472106b7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:42.950388 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddead1-c768-48b7-ac90-e788472106b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:42.950665 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ddead1-c768-48b7-ac90-e788472106b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:42.950930 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:42.951433 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ddead1-c768-48b7-ac90-e788472106b7 None admin] RBAC: Adding query filter params (role.id=f886b8e8889549e896a8fb3426cce9e1, scope.project.id=d18a5117a03b4f21ab3290a15680de66, user.id=30fe233a7d4c4fbea6ede233bd9ff8c8) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:42.951757 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ddead1-c768-48b7-ac90-e788472106b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:42.953133 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ddead1-c768-48b7-ac90-e788472106b7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:42.966745 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 379/770] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:14:42 2022] GET /identity/v3/role_assignments?user.id=30fe233a7d4c4fbea6ede233bd9ff8c8&scope.project.id=d18a5117a03b4f21ab3290a15680de66&role.id=f886b8e8889549e896a8fb3426cce9e1 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:42.966745 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddead1-c768-48b7-ac90-e788472106b7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:43.046568 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e437d7-69c3-41e5-a83b-de7dd6d60981 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:43.086727 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36c1c7-6878-4f28-bde5-766848e14512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:43.087412 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36c1c7-6878-4f28-bde5-766848e14512 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:43.087695 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36c1c7-6878-4f28-bde5-766848e14512 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:43.087965 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36c1c7-6878-4f28-bde5-766848e14512 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:43.089580 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ddead1-c768-48b7-ac90-e788472106b7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:43.097026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf36c1c7-6878-4f28-bde5-766848e14512 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:43.097549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf36c1c7-6878-4f28-bde5-766848e14512 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:43.112217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-cf36c1c7-6878-4f28-bde5-766848e14512 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:43.112217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:43.112217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:43.112217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:43.112217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:43.112217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:43.112217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:43.112217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:43.112217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:43.112217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:43.112217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:43.112217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:43.112217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:14:43.112217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:14:43.112217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:14:43.113165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:14:43.113165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:43.113165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:43.113165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:43.113165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:43.113165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:43.113165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:43.113165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:43.113165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:43.113165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:43.113165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:43.113165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:43.113165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:43.113165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:43.113165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:43.113165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:43.113165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:43.114065 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:14:43.114065 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:14:43.114065 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:14:43.114065 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:14:43.114065 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:14:43.114065 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:14:43.114065 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:43.114065 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:43.114065 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 392/771] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:43.114565 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36c1c7-6878-4f28-bde5-766848e14512 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:43.133160 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459145d7-8997-49fd-b020-e119c76234df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:43.133805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459145d7-8997-49fd-b020-e119c76234df None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:43.134087 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459145d7-8997-49fd-b020-e119c76234df None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:43.134363 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459145d7-8997-49fd-b020-e119c76234df None admin] PATH_INFO: `/v3/services/network` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:43.134972 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459145d7-8997-49fd-b020-e119c76234df None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:43.136492 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459145d7-8997-49fd-b020-e119c76234df None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:43.145477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-459145d7-8997-49fd-b020-e119c76234df None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 02 09:14:43.145477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:43.145477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 09:14:43.145477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 09:14:43.145477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 09:14:43.145477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 09:14:43.145477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 09:14:43.145477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:14:43.145477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 02 09:14:43.145477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:43.145477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:14:43.145477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:43.145477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:43.145477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:43.145477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:43.145477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:43.145477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:43.147018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:43.147018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:43.147018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:43.147018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:43.147018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:43.147018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:43.147018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 09:14:43.147018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 09:14:43.147018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 09:14:43.147018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 09:14:43.147018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:43.147018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:43.147018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:43.147018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:43.147018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:43.147018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:43.147859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:43.147859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:43.147859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:43.147859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:43.147859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:43.147859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:43.147859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:43.147859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:43.147859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:43.147859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:43.147859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 09:14:43.147859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:14:43.147859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 02 09:14:43.147859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:43.151538 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 380/772] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:42 2022] GET /identity/v3/services/network => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:43.152398 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459145d7-8997-49fd-b020-e119c76234df None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:43.202479 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:43.214923 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:43.232046 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36c1c7-6878-4f28-bde5-766848e14512 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:43.259353 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459145d7-8997-49fd-b020-e119c76234df None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:43.269641 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e22f335-3631-4c8e-ab00-a0cfaa33a2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:43.270566 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22f335-3631-4c8e-ab00-a0cfaa33a2d9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:43.272224 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22f335-3631-4c8e-ab00-a0cfaa33a2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:43.272949 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22f335-3631-4c8e-ab00-a0cfaa33a2d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:43.273852 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e22f335-3631-4c8e-ab00-a0cfaa33a2d9 None admin] RBAC: Adding query filter params (interface=public, service_id=5dc199dbac524f63863d8023d939dc0a, region_id=RegionOne) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:43.274604 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e22f335-3631-4c8e-ab00-a0cfaa33a2d9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:43.276962 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e22f335-3631-4c8e-ab00-a0cfaa33a2d9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:43.294810 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 393/773] 10.222.0.57 () {56 vars in 1251 bytes} [Fri Sep 2 09:14:43 2022] GET /identity/v3/endpoints?service_id=5dc199dbac524f63863d8023d939dc0a&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:43.296378 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e22f335-3631-4c8e-ab00-a0cfaa33a2d9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:43.297259 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a389582-f757-4df4-9f9b-b098e21b318a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:43.297878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a389582-f757-4df4-9f9b-b098e21b318a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:43.298160 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a389582-f757-4df4-9f9b-b098e21b318a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:43.298434 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a389582-f757-4df4-9f9b-b098e21b318a None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:43.299173 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a389582-f757-4df4-9f9b-b098e21b318a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:43.299481 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a389582-f757-4df4-9f9b-b098e21b318a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:43.301005 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a389582-f757-4df4-9f9b-b098e21b318a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:43.319285 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 381/774] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:14:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:43.319285 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a389582-f757-4df4-9f9b-b098e21b318a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:43.405890 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e22f335-3631-4c8e-ab00-a0cfaa33a2d9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:43.426343 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a389582-f757-4df4-9f9b-b098e21b318a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:43.433963 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:43.444842 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ae8d9-55e5-488f-8de1-a1265842274b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:43.445838 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ae8d9-55e5-488f-8de1-a1265842274b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:43.446285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ae8d9-55e5-488f-8de1-a1265842274b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:43.460147 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ae8d9-55e5-488f-8de1-a1265842274b None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:43.460147 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340ae8d9-55e5-488f-8de1-a1265842274b None admin] RBAC: Adding query filter params (name=network) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:43.460147 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340ae8d9-55e5-488f-8de1-a1265842274b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:43.460147 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340ae8d9-55e5-488f-8de1-a1265842274b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:43.460147 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 394/775] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Sep 2 09:14:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:43.460147 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ae8d9-55e5-488f-8de1-a1265842274b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:43.469560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d99d2ff-d050-418f-9c8e-11c76762ac7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:43.470241 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99d2ff-d050-418f-9c8e-11c76762ac7d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:43.470558 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99d2ff-d050-418f-9c8e-11c76762ac7d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:43.470872 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99d2ff-d050-418f-9c8e-11c76762ac7d None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:43.471491 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d99d2ff-d050-418f-9c8e-11c76762ac7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:43.472712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:43.472870 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d99d2ff-d050-418f-9c8e-11c76762ac7d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:43.479290 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 382/776] 10.222.0.57 () {56 vars in 1087 bytes} [Fri Sep 2 09:14:43 2022] GET /identity/v3/services => generated 772 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:43.486768 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d99d2ff-d050-418f-9c8e-11c76762ac7d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:43.593186 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d99d2ff-d050-418f-9c8e-11c76762ac7d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:43.627370 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340ae8d9-55e5-488f-8de1-a1265842274b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:43.631893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29caca7-e5ae-4390-9817-c5282b89910d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:43.632523 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29caca7-e5ae-4390-9817-c5282b89910d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:43.632809 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29caca7-e5ae-4390-9817-c5282b89910d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:43.633088 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29caca7-e5ae-4390-9817-c5282b89910d None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:43.633734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29caca7-e5ae-4390-9817-c5282b89910d None admin] RBAC: Adding query filter params (type=network) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:43.634044 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29caca7-e5ae-4390-9817-c5282b89910d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:43.635483 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29caca7-e5ae-4390-9817-c5282b89910d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:43.647252 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 383/777] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Sep 2 09:14:43 2022] GET /identity/v3/services?type=network => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:43.649299 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29caca7-e5ae-4390-9817-c5282b89910d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:43.665456 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a4bdae-0493-4b2f-b7c3-da037222f00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:43.666076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4bdae-0493-4b2f-b7c3-da037222f00c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:43.666358 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4bdae-0493-4b2f-b7c3-da037222f00c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:43.666630 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a4bdae-0493-4b2f-b7c3-da037222f00c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:43.667398 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a4bdae-0493-4b2f-b7c3-da037222f00c None admin] RBAC: Adding query filter params (role.id=faf1138f11d34abe8a862f1a1f7c140a, scope.project.id=d18a5117a03b4f21ab3290a15680de66, user.id=5233689a53364258b86ad2fa012d8db1) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:43.667710 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a4bdae-0493-4b2f-b7c3-da037222f00c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:43.669150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a4bdae-0493-4b2f-b7c3-da037222f00c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:43.675191 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 395/778] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:14:43 2022] GET /identity/v3/role_assignments?user.id=5233689a53364258b86ad2fa012d8db1&scope.project.id=d18a5117a03b4f21ab3290a15680de66&role.id=faf1138f11d34abe8a862f1a1f7c140a => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:43.690122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:43.730908 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1699d8-8114-42eb-9281-9cc7d05f3b3c None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:43.731665 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1699d8-8114-42eb-9281-9cc7d05f3b3c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:43.732157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1699d8-8114-42eb-9281-9cc7d05f3b3c None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:43.734160 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 396/779] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:14:43.740345 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d2e83-173c-410d-bbd3-2d5d13cd040d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:43.740966 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d2e83-173c-410d-bbd3-2d5d13cd040d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:43.741474 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0d2e83-173c-410d-bbd3-2d5d13cd040d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:43.763490 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9c0d2e83-173c-410d-bbd3-2d5d13cd040d None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:43.765136 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9c0d2e83-173c-410d-bbd3-2d5d13cd040d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:43.773488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29caca7-e5ae-4390-9817-c5282b89910d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:43.810193 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1de94c-d147-4330-a7bd-d26ca9a9ba66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:43.810860 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1de94c-d147-4330-a7bd-d26ca9a9ba66 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:43.811222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1de94c-d147-4330-a7bd-d26ca9a9ba66 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:43.811503 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1de94c-d147-4330-a7bd-d26ca9a9ba66 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:43.812390 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1de94c-d147-4330-a7bd-d26ca9a9ba66 None admin] RBAC: Adding query filter params (interface=public, service_id=08527b54fc994606a0aa2185f037197d, region_id=RegionOne) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:43.812746 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1de94c-d147-4330-a7bd-d26ca9a9ba66 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:43.814196 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1de94c-d147-4330-a7bd-d26ca9a9ba66 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:43.828254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 384/780] 10.222.0.57 () {56 vars in 1251 bytes} [Fri Sep 2 09:14:43 2022] GET /identity/v3/endpoints?service_id=08527b54fc994606a0aa2185f037197d&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:43.829666 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1de94c-d147-4330-a7bd-d26ca9a9ba66 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:43.863328 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 397/781] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 09:14:43.935934 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:43.940282 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1de94c-d147-4330-a7bd-d26ca9a9ba66 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:43.978028 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1380d3e2-1874-474b-bfb9-77c1cd4c0e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:43.978651 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380d3e2-1874-474b-bfb9-77c1cd4c0e2d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:43.978959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380d3e2-1874-474b-bfb9-77c1cd4c0e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:43.979236 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380d3e2-1874-474b-bfb9-77c1cd4c0e2d None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:43.979843 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1380d3e2-1874-474b-bfb9-77c1cd4c0e2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:43.981335 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1380d3e2-1874-474b-bfb9-77c1cd4c0e2d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:43.990828 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 385/782] 10.222.0.57 () {56 vars in 1087 bytes} [Fri Sep 2 09:14:43 2022] GET /identity/v3/services => generated 772 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:44.145676 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d2e83-173c-410d-bbd3-2d5d13cd040d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:44.155175 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:44.254386 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0d2e83-173c-410d-bbd3-2d5d13cd040d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:44.412484 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a4960-4e49-4689-8698-77b9366b4f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:44.413108 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a4960-4e49-4689-8698-77b9366b4f96 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:44.413377 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a4960-4e49-4689-8698-77b9366b4f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:44.413640 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a4960-4e49-4689-8698-77b9366b4f96 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:44.424048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a4960-4e49-4689-8698-77b9366b4f96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:44.429689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-174a4960-4e49-4689-8698-77b9366b4f96 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:14:44.429689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:44.429689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:44.429689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:44.429689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:44.429689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:44.429689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:44.429689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:44.429689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:44.429689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:44.429689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:44.429689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:44.429689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:44.429689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:44.429689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:44.429689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:44.430610 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:44.430610 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:44.430610 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:44.430610 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:44.430610 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:44.430610 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:44.430610 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:44.430610 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:44.430610 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:44.430610 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:44.430610 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:44.430610 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:44.430610 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:44.430610 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:44.430610 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:44.430610 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:44.431486 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:44.431486 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:44.431486 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:44.431486 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:44.431486 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:44.431486 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:44.431486 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:44.431486 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:44.431486 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:44.431486 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:44.431486 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:14:44.431486 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:44.431486 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 398/783] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:14:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:44.436882 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1380d3e2-1874-474b-bfb9-77c1cd4c0e2d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:44.570988 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1380d3e2-1874-474b-bfb9-77c1cd4c0e2d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:44.689848 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b0d84-1d17-43ca-987f-82fc80cd43ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:44.690467 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b0d84-1d17-43ca-987f-82fc80cd43ff None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:44.690765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b0d84-1d17-43ca-987f-82fc80cd43ff None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:44.691046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b0d84-1d17-43ca-987f-82fc80cd43ff None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:44.691711 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b0d84-1d17-43ca-987f-82fc80cd43ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:44.692014 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b0d84-1d17-43ca-987f-82fc80cd43ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:44.693904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348b0d84-1d17-43ca-987f-82fc80cd43ff None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:44.707081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 386/784] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:14:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:44.712861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a4960-4e49-4689-8698-77b9366b4f96 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:44.806482 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:44.854905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a4960-4e49-4689-8698-77b9366b4f96 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:44.889426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557bca18-6e6f-403a-8d7e-3d328ab39dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:44.890053 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557bca18-6e6f-403a-8d7e-3d328ab39dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:44.890327 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557bca18-6e6f-403a-8d7e-3d328ab39dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:44.890584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557bca18-6e6f-403a-8d7e-3d328ab39dc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:44.901145 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557bca18-6e6f-403a-8d7e-3d328ab39dc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:44.902560 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557bca18-6e6f-403a-8d7e-3d328ab39dc1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:44.916359 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-557bca18-6e6f-403a-8d7e-3d328ab39dc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:44.916359 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:44.916359 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:44.916359 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:44.916359 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:44.916359 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:44.916359 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:44.916359 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:44.916359 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:44.916359 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:44.916359 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:44.916359 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:44.916359 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:44.916359 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:44.916359 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:44.916359 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:44.917314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:44.917314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:44.917314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:44.917314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:44.917314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:44.917314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:44.917314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:44.917314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:44.917314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:44.917314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:44.917314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:44.917314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:44.917314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:44.917314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:44.917314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:44.917314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:44.917314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:44.918224 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:44.918224 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:44.918224 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:44.918224 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 399/785] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:44.923358 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b0d84-1d17-43ca-987f-82fc80cd43ff None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:44.959126 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:45.042418 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348b0d84-1d17-43ca-987f-82fc80cd43ff None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:45.092582 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f03b2a-a06c-45a5-be86-78cae0140598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:45.093199 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f03b2a-a06c-45a5-be86-78cae0140598 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:45.093476 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f03b2a-a06c-45a5-be86-78cae0140598 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:45.093752 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f03b2a-a06c-45a5-be86-78cae0140598 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:45.094453 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f03b2a-a06c-45a5-be86-78cae0140598 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:45.094694 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f03b2a-a06c-45a5-be86-78cae0140598 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:45.096197 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f03b2a-a06c-45a5-be86-78cae0140598 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:45.114644 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 387/786] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:45.120941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557bca18-6e6f-403a-8d7e-3d328ab39dc1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:45.145997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e65fc-1e2e-427b-9eec-59472263b84a None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:45.146350 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e65fc-1e2e-427b-9eec-59472263b84a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:45.146566 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e65fc-1e2e-427b-9eec-59472263b84a None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:45.148289 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 388/787] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:45.157046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977967c6-41ea-4789-94f4-f0cef039d11f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:45.157327 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977967c6-41ea-4789-94f4-f0cef039d11f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:45.157599 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977967c6-41ea-4789-94f4-f0cef039d11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:45.187756 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-977967c6-41ea-4789-94f4-f0cef039d11f None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:45.188988 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-977967c6-41ea-4789-94f4-f0cef039d11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:45.287642 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557bca18-6e6f-403a-8d7e-3d328ab39dc1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:45.307833 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 389/788] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:14:45.323108 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:45.331305 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146636c9-f17c-40ed-a335-2b7593ee9826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:45.331948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146636c9-f17c-40ed-a335-2b7593ee9826 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:45.332228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146636c9-f17c-40ed-a335-2b7593ee9826 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:45.332500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146636c9-f17c-40ed-a335-2b7593ee9826 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:45.340078 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146636c9-f17c-40ed-a335-2b7593ee9826 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:45.348599 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146636c9-f17c-40ed-a335-2b7593ee9826 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:45.352922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-146636c9-f17c-40ed-a335-2b7593ee9826 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 02 09:14:45.352922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:45.352922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:45.352922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:45.352922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:45.352922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:45.352922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:45.352922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:45.352922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:45.352922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:45.352922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:45.352922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:45.352922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:14:45.352922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:14:45.352922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:14:45.352922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:14:45.353866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:45.353866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:45.353866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:14:45.353866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:45.353866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:14:45.353866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:45.353866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:45.353866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:45.353866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:45.353866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:45.353866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:45.353866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:45.353866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:45.353866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:45.353866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:45.353866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:45.353866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:45.353866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:45.355367 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:45.355367 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:45.355367 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:14:45.355367 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:14:45.355367 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:14:45.355367 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:14:45.355367 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:14:45.355367 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:14:45.355367 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 02 09:14:45.355367 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:45.355367 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 400/789] 10.222.0.57 () {56 vars in 1093 bytes} [Fri Sep 2 09:14:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:45.359798 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977967c6-41ea-4789-94f4-f0cef039d11f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:45.386379 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:45.503440 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977967c6-41ea-4789-94f4-f0cef039d11f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:45.556095 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d0fce5-21b3-4573-94a1-9af7e77fe9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:45.557170 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d0fce5-21b3-4573-94a1-9af7e77fe9b9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:45.557654 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d0fce5-21b3-4573-94a1-9af7e77fe9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:45.558095 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d0fce5-21b3-4573-94a1-9af7e77fe9b9 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:45.559014 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d0fce5-21b3-4573-94a1-9af7e77fe9b9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:45.559490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d0fce5-21b3-4573-94a1-9af7e77fe9b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:45.561272 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d0fce5-21b3-4573-94a1-9af7e77fe9b9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:45.586629 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 390/790] 10.222.0.57 () {56 vars in 1133 bytes} [Fri Sep 2 09:14:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:45.592276 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146636c9-f17c-40ed-a335-2b7593ee9826 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:45.754947 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146636c9-f17c-40ed-a335-2b7593ee9826 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:45.785022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d0fce5-21b3-4573-94a1-9af7e77fe9b9 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:45.811073 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0f6da-0852-4d31-bb8b-815e471e3fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:45.811695 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0f6da-0852-4d31-bb8b-815e471e3fc8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:45.811977 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0f6da-0852-4d31-bb8b-815e471e3fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:45.812366 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d0f6da-0852-4d31-bb8b-815e471e3fc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:45.876193 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:45.903428 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:45.938725 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d0fce5-21b3-4573-94a1-9af7e77fe9b9 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:45.949948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d0f6da-0852-4d31-bb8b-815e471e3fc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:45.951561 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d0f6da-0852-4d31-bb8b-815e471e3fc8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:45.990876 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b9623-37ca-4b5e-9a2d-38aa6de9c177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:45.991519 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b9623-37ca-4b5e-9a2d-38aa6de9c177 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:45.991794 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b9623-37ca-4b5e-9a2d-38aa6de9c177 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:45.992061 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b9623-37ca-4b5e-9a2d-38aa6de9c177 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:46.020722 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b9623-37ca-4b5e-9a2d-38aa6de9c177 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:46.025525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-da2b9623-37ca-4b5e-9a2d-38aa6de9c177 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:14:46.025525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:46.025525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:46.025525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:46.025525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:46.025525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:46.025525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:46.025525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:46.025525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:46.025525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:46.025525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:46.025525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:46.025525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:46.025525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:46.025525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:46.025525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:46.026629 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:46.026629 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:46.026629 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:46.026629 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:46.026629 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:46.026629 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:46.026629 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:46.026629 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:46.026629 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:46.026629 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:46.026629 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:46.026629 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:46.026629 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:46.026629 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:46.026629 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:46.026629 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:46.027684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:46.027684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:46.027684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:46.027684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:46.027684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:46.027684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:46.027684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:46.027684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:46.027684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:46.027684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:46.027684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:14:46.027684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:46.038290 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 391/791] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:14:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:46.043056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b9623-37ca-4b5e-9a2d-38aa6de9c177 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:46.146739 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 401/792] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:14:45 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 09:14:46.147645 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0f6da-0852-4d31-bb8b-815e471e3fc8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:46.224222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b9623-37ca-4b5e-9a2d-38aa6de9c177 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:46.241514 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:46.258211 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:46.263362 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc1640-fd0a-48b3-9237-3916f38a82d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:46.264220 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc1640-fd0a-48b3-9237-3916f38a82d9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:46.264718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc1640-fd0a-48b3-9237-3916f38a82d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:46.264825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d0f6da-0852-4d31-bb8b-815e471e3fc8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:46.265314 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc1640-fd0a-48b3-9237-3916f38a82d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:46.266206 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc1640-fd0a-48b3-9237-3916f38a82d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:46.266729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc1640-fd0a-48b3-9237-3916f38a82d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:46.268353 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc1640-fd0a-48b3-9237-3916f38a82d9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:46.278637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 392/793] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:14:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:46.280223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f73bf9-27a5-4b09-9134-0cac52ef046b None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:46.280758 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f73bf9-27a5-4b09-9134-0cac52ef046b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:46.280927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f73bf9-27a5-4b09-9134-0cac52ef046b None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:46.282060 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 393/794] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:46.283122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca33176-e3ad-4cb3-9812-fc208b560261 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:46.283400 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca33176-e3ad-4cb3-9812-fc208b560261 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:46.283669 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca33176-e3ad-4cb3-9812-fc208b560261 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:46.286174 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 394/795] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:46.286903 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca33176-e3ad-4cb3-9812-fc208b560261 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:46.306517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab191e91-d1da-46ee-82e5-01ca8061ae3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:46.307264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab191e91-d1da-46ee-82e5-01ca8061ae3e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:46.307552 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab191e91-d1da-46ee-82e5-01ca8061ae3e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:46.307825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab191e91-d1da-46ee-82e5-01ca8061ae3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:46.317853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab191e91-d1da-46ee-82e5-01ca8061ae3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:46.319374 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab191e91-d1da-46ee-82e5-01ca8061ae3e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:46.334555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ab191e91-d1da-46ee-82e5-01ca8061ae3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:46.334555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:46.334555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:46.334555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:46.334555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:46.334555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:46.334555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:46.334555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:46.334555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:46.334555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:46.334555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:46.334555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:46.334555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:46.334555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:46.334555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:46.334555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:46.335678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:46.335678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:46.335678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:46.335678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:46.335678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:46.335678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:46.335678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:46.335678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:46.335678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:46.335678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:46.335678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:46.335678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:46.335678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:46.335678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:46.335678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:46.335678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:46.335678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:46.336556 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:46.336556 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:46.336556 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:46.336556 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 402/796] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:46.337171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b68e5d-a242-4251-a863-e6708c2f0b1e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:46.337171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b68e5d-a242-4251-a863-e6708c2f0b1e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:46.337411 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b68e5d-a242-4251-a863-e6708c2f0b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:46.355096 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b8b68e5d-a242-4251-a863-e6708c2f0b1e None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:46.356424 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b8b68e5d-a242-4251-a863-e6708c2f0b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:46.392976 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca33176-e3ad-4cb3-9812-fc208b560261 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:46.426913 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5268f4a-9c9b-4715-8575-4afb047e9577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:46.427563 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5268f4a-9c9b-4715-8575-4afb047e9577 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:46.427844 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5268f4a-9c9b-4715-8575-4afb047e9577 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:46.428125 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5268f4a-9c9b-4715-8575-4afb047e9577 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:46.428336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 403/797] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:14:46.429597 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da143451-fe08-448f-b728-afd2ea84740f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:46.429893 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da143451-fe08-448f-b728-afd2ea84740f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:46.430160 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da143451-fe08-448f-b728-afd2ea84740f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:46.438396 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5268f4a-9c9b-4715-8575-4afb047e9577 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:46.439852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5268f4a-9c9b-4715-8575-4afb047e9577 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:46.456871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-da143451-fe08-448f-b728-afd2ea84740f None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:46.457178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-c5268f4a-9c9b-4715-8575-4afb047e9577 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:46.457178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:46.457178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:46.457178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:46.457178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:46.457178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:46.457178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:46.457178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:46.457178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:46.457178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:46.457178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:46.457178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:46.457178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:46.457178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:46.457178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:46.457178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:46.458068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:46.458068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:46.458068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:46.458068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:46.458068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:46.458068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:46.458068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:46.458068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:46.458068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:46.458068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:46.458068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:46.458068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:46.458068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:46.458068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:46.458068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:46.458068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:46.458068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:46.464785 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:46.464785 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:46.464785 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:46.464980 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-da143451-fe08-448f-b728-afd2ea84740f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:46.465059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 395/798] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:46.465059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda64e94-36be-4119-8b67-2c17194c9a12 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:46.465059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda64e94-36be-4119-8b67-2c17194c9a12 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:46.465059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda64e94-36be-4119-8b67-2c17194c9a12 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:46.465059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 396/799] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:46.465059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda64e94-36be-4119-8b67-2c17194c9a12 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:46.512979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:46.523305 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:46.534560 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 404/800] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:14:46.539730 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da143451-fe08-448f-b728-afd2ea84740f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:46.573498 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda64e94-36be-4119-8b67-2c17194c9a12 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:46.610842 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f84e081-5c8e-48d6-8360-a7a5ed52353b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:46.611748 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84e081-5c8e-48d6-8360-a7a5ed52353b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:46.612284 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84e081-5c8e-48d6-8360-a7a5ed52353b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:46.612705 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f84e081-5c8e-48d6-8360-a7a5ed52353b None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:46.613641 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84e081-5c8e-48d6-8360-a7a5ed52353b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:46.614094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84e081-5c8e-48d6-8360-a7a5ed52353b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:46.615808 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f84e081-5c8e-48d6-8360-a7a5ed52353b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:46.628670 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 397/801] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:46.631936 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339d2aa-2aae-45c5-af22-d77d728a6267 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:46.632437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339d2aa-2aae-45c5-af22-d77d728a6267 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:46.632876 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339d2aa-2aae-45c5-af22-d77d728a6267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:46.657728 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a339d2aa-2aae-45c5-af22-d77d728a6267 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:46.659250 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a339d2aa-2aae-45c5-af22-d77d728a6267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:46.668452 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da143451-fe08-448f-b728-afd2ea84740f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:46.706974 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1433405a-67c1-4531-be12-890b621e54c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:46.707953 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433405a-67c1-4531-be12-890b621e54c1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:46.708513 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433405a-67c1-4531-be12-890b621e54c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:46.709537 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433405a-67c1-4531-be12-890b621e54c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:46.710828 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1433405a-67c1-4531-be12-890b621e54c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:46.711362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1433405a-67c1-4531-be12-890b621e54c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:46.713339 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1433405a-67c1-4531-be12-890b621e54c1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:46.725152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 405/802] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:46.726390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1433405a-67c1-4531-be12-890b621e54c1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:46.739359 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 398/803] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:14:46.740403 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339d2aa-2aae-45c5-af22-d77d728a6267 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:46.778844 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:46.797167 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:46.854999 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1433405a-67c1-4531-be12-890b621e54c1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:46.860255 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339d2aa-2aae-45c5-af22-d77d728a6267 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:46.896902 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a1dcc-a470-4b92-a10e-cc1802413561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:46.897541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a1dcc-a470-4b92-a10e-cc1802413561 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:46.897850 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a1dcc-a470-4b92-a10e-cc1802413561 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:46.898123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a1dcc-a470-4b92-a10e-cc1802413561 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:46.898696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707a1dcc-a470-4b92-a10e-cc1802413561 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:46.900093 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707a1dcc-a470-4b92-a10e-cc1802413561 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:46.902850 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e462b1a7-5d74-42d7-974f-874ef0be7b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:46.903946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e462b1a7-5d74-42d7-974f-874ef0be7b02 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:46.904374 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e462b1a7-5d74-42d7-974f-874ef0be7b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:46.904801 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e462b1a7-5d74-42d7-974f-874ef0be7b02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:46.908846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-707a1dcc-a470-4b92-a10e-cc1802413561 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 02 09:14:46.908846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:46.908846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 09:14:46.908846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 09:14:46.908846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 09:14:46.908846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 09:14:46.908846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 09:14:46.908846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:14:46.908846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 02 09:14:46.908846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:46.908846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:14:46.908846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:46.908846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:46.908846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:46.908846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:46.908846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:46.908846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:46.910041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:46.910041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:46.910041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:46.910041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:46.910041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:46.910041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:46.910041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 09:14:46.910041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 09:14:46.910041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 09:14:46.910041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 09:14:46.910041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:46.910041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:46.910041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:46.910041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:46.910041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:46.910041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:46.910911 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:46.910911 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:46.910911 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:46.910911 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:46.910911 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:46.910911 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:46.910911 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:46.910911 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:46.910911 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:46.910911 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:46.910911 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 09:14:46.910911 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:14:46.910911 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 02 09:14:46.910911 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:46.910911 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 399/804] 10.222.0.57 () {56 vars in 1129 bytes} [Fri Sep 2 09:14:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:46.911697 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a1dcc-a470-4b92-a10e-cc1802413561 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:46.915959 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e462b1a7-5d74-42d7-974f-874ef0be7b02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:46.918784 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e462b1a7-5d74-42d7-974f-874ef0be7b02 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:46.931653 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-e462b1a7-5d74-42d7-974f-874ef0be7b02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:46.931653 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:46.931653 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:46.931653 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:46.931653 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:46.931653 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:46.931653 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:46.931653 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:46.931653 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:46.931653 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:46.931653 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:46.931653 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:46.931653 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:14:46.931653 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:14:46.931653 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:14:46.932712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:14:46.932712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:46.932712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:46.932712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:46.932712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:46.932712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:46.932712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:46.932712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:46.932712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:46.932712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:46.932712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:46.932712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:46.932712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:46.932712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:46.932712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:46.932712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:46.932712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:46.938592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:14:46.938592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:14:46.938592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:14:46.938592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:14:46.938592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:14:46.938592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:14:46.938592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:46.938592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:46.938592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 406/805] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:46.938592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e462b1a7-5d74-42d7-974f-874ef0be7b02 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:47.019489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:47.023209 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a1dcc-a470-4b92-a10e-cc1802413561 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:47.037660 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e462b1a7-5d74-42d7-974f-874ef0be7b02 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:47.047767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:47.057311 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d06ea-2338-41f9-8123-fb5e95bd5c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:47.057978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d06ea-2338-41f9-8123-fb5e95bd5c3e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:47.058255 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d06ea-2338-41f9-8123-fb5e95bd5c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:47.058529 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d06ea-2338-41f9-8123-fb5e95bd5c3e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:47.065808 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d06ea-2338-41f9-8123-fb5e95bd5c3e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:47.067340 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d06ea-2338-41f9-8123-fb5e95bd5c3e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:47.072414 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46162b3-1802-46d5-9948-4e7f6ee63ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:47.073058 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46162b3-1802-46d5-9948-4e7f6ee63ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:47.073527 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46162b3-1802-46d5-9948-4e7f6ee63ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:47.073863 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46162b3-1802-46d5-9948-4e7f6ee63ff9 None admin] PATH_INFO: `/v3/services/network` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:47.074596 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46162b3-1802-46d5-9948-4e7f6ee63ff9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:47.076246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46162b3-1802-46d5-9948-4e7f6ee63ff9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:47.079324 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-c80d06ea-2338-41f9-8123-fb5e95bd5c3e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 02 09:14:47.079324 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:47.079324 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:47.079324 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:47.079324 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:47.079324 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:47.079324 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:47.079324 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:47.079324 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:47.079324 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:47.079324 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:47.079324 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:47.079324 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:14:47.079324 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:14:47.079324 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:14:47.079324 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:14:47.080432 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:47.080432 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:47.080432 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:14:47.080432 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:47.080432 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:14:47.080432 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:47.080432 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:47.080432 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:47.080432 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:47.080432 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:47.080432 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:47.080432 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:47.080432 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:47.080432 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:47.080432 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:47.080432 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:47.080432 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:47.080432 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:47.081940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:47.081940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:47.081940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:14:47.081940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:14:47.081940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:14:47.081940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:14:47.081940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:14:47.081940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:14:47.081940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 02 09:14:47.081940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:47.081940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 400/806] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:14:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:47.081940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d06ea-2338-41f9-8123-fb5e95bd5c3e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:47.086816 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-c46162b3-1802-46d5-9948-4e7f6ee63ff9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 02 09:14:47.086816 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:47.086816 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 09:14:47.086816 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 09:14:47.086816 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 09:14:47.086816 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 09:14:47.086816 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 09:14:47.086816 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:14:47.086816 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 02 09:14:47.086816 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:47.086816 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:14:47.086816 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:47.086816 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:47.086816 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:47.086816 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:47.086816 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:47.086816 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:47.087792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:47.087792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:47.087792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:47.087792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:47.087792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:47.087792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:47.087792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 09:14:47.087792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 09:14:47.087792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 09:14:47.087792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 09:14:47.087792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:47.087792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:47.087792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:47.087792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:47.087792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:47.087792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:47.088608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:47.088608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:47.088608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:47.088608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:47.088608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:47.088608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:47.088608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:47.088608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:47.088608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:47.088608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:47.088608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 09:14:47.088608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:14:47.088608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 02 09:14:47.088608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:47.088608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 407/807] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:46 2022] GET /identity/v3/services/network => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:47.090811 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46162b3-1802-46d5-9948-4e7f6ee63ff9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:47.182274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d06ea-2338-41f9-8123-fb5e95bd5c3e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:47.186612 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46162b3-1802-46d5-9948-4e7f6ee63ff9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:47.215693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7310af-2511-48c9-a1cd-49e6f03e99cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:47.216295 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7310af-2511-48c9-a1cd-49e6f03e99cb None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:47.216580 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7310af-2511-48c9-a1cd-49e6f03e99cb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:47.216856 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7310af-2511-48c9-a1cd-49e6f03e99cb None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:47.217480 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7310af-2511-48c9-a1cd-49e6f03e99cb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:47.217784 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7310af-2511-48c9-a1cd-49e6f03e99cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:47.219181 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7310af-2511-48c9-a1cd-49e6f03e99cb None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:47.219765 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e7a73-f2ac-4191-88ce-cd8dccd3e5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:47.220366 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e7a73-f2ac-4191-88ce-cd8dccd3e5a8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:47.220644 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e7a73-f2ac-4191-88ce-cd8dccd3e5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:47.220922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e7a73-f2ac-4191-88ce-cd8dccd3e5a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:47.221608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e7a73-f2ac-4191-88ce-cd8dccd3e5a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:47.221912 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e7a73-f2ac-4191-88ce-cd8dccd3e5a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:47.223340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17e7a73-f2ac-4191-88ce-cd8dccd3e5a8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:47.223742 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 401/808] 10.222.0.57 () {56 vars in 1124 bytes} [Fri Sep 2 09:14:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:47.224554 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7310af-2511-48c9-a1cd-49e6f03e99cb None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:47.239236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 408/809] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:14:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:47.240581 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e7a73-f2ac-4191-88ce-cd8dccd3e5a8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:47.251148 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:47.274055 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:47.325091 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7310af-2511-48c9-a1cd-49e6f03e99cb None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:47.337272 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17e7a73-f2ac-4191-88ce-cd8dccd3e5a8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:47.358188 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa8e80-4269-439c-93c8-04c4c4586e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:47.358844 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa8e80-4269-439c-93c8-04c4c4586e42 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:47.359122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa8e80-4269-439c-93c8-04c4c4586e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:47.359394 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaa8e80-4269-439c-93c8-04c4c4586e42 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:47.369991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaa8e80-4269-439c-93c8-04c4c4586e42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:47.372107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afddbef4-4ff3-4b3a-960d-93cc6b85838b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:47.372709 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afddbef4-4ff3-4b3a-960d-93cc6b85838b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:47.372991 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afddbef4-4ff3-4b3a-960d-93cc6b85838b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:47.373266 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afddbef4-4ff3-4b3a-960d-93cc6b85838b None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:47.374018 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afddbef4-4ff3-4b3a-960d-93cc6b85838b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:47.374326 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afddbef4-4ff3-4b3a-960d-93cc6b85838b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:47.375218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-eaaa8e80-4269-439c-93c8-04c4c4586e42 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:47.375218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:47.375218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:47.375218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:47.375218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:47.375218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:47.375218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:47.375218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:47.375218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:47.375218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:47.375218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:47.375218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:47.375218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:47.375218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:47.375218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:47.375218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:47.376280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:47.376280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:47.376280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:47.376280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:47.376280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:47.376280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:47.376280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:47.376280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:47.376280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:47.376280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:47.376280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:47.376280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:47.376280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:47.376280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:47.376280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:47.376280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:47.377105 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:47.377105 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:47.377105 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:47.377105 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:47.377105 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:47.377105 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:47.377105 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:47.377105 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:47.377105 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:47.377105 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:47.377105 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:47.377105 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:47.377759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afddbef4-4ff3-4b3a-960d-93cc6b85838b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:47.377829 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 402/810] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Sep 2 09:14:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:47.377829 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa8e80-4269-439c-93c8-04c4c4586e42 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:47.394589 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 409/811] 10.222.0.57 () {56 vars in 1135 bytes} [Fri Sep 2 09:14:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:47.395435 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afddbef4-4ff3-4b3a-960d-93cc6b85838b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:47.466613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:47.482499 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaa8e80-4269-439c-93c8-04c4c4586e42 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:47.484837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:47.493948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afddbef4-4ff3-4b3a-960d-93cc6b85838b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:47.515778 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9a8d6-8fb2-430d-a46f-46c96e3dcc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:47.516377 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9a8d6-8fb2-430d-a46f-46c96e3dcc3b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:47.516652 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9a8d6-8fb2-430d-a46f-46c96e3dcc3b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:47.516919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9a8d6-8fb2-430d-a46f-46c96e3dcc3b None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:47.517540 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9a8d6-8fb2-430d-a46f-46c96e3dcc3b None admin] RBAC: Adding query filter params (name=network) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:47.517838 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9a8d6-8fb2-430d-a46f-46c96e3dcc3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:47.519233 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9a8d6-8fb2-430d-a46f-46c96e3dcc3b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:47.523447 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 403/812] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Sep 2 09:14:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:47.524237 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9a8d6-8fb2-430d-a46f-46c96e3dcc3b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:47.526085 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4418c-3d86-4e80-9f69-9cd02481f6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:47.526692 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4418c-3d86-4e80-9f69-9cd02481f6b7 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:47.527001 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4418c-3d86-4e80-9f69-9cd02481f6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:47.527275 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4418c-3d86-4e80-9f69-9cd02481f6b7 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:47.527910 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4418c-3d86-4e80-9f69-9cd02481f6b7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:47.528447 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4418c-3d86-4e80-9f69-9cd02481f6b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:47.529794 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd4418c-3d86-4e80-9f69-9cd02481f6b7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:47.535239 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 410/813] 10.222.0.57 () {56 vars in 1124 bytes} [Fri Sep 2 09:14:47 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:47.536009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4418c-3d86-4e80-9f69-9cd02481f6b7 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:47.627854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9a8d6-8fb2-430d-a46f-46c96e3dcc3b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:47.636396 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4418c-3d86-4e80-9f69-9cd02481f6b7 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:47.660434 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c213d1-df7d-4a0d-a27e-f1f6a1a06af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:47.661029 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c213d1-df7d-4a0d-a27e-f1f6a1a06af2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:47.661310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c213d1-df7d-4a0d-a27e-f1f6a1a06af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:47.661581 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c213d1-df7d-4a0d-a27e-f1f6a1a06af2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:47.662852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c213d1-df7d-4a0d-a27e-f1f6a1a06af2 None admin] RBAC: Adding query filter params (role.id=f886b8e8889549e896a8fb3426cce9e1, scope.project.id=d18a5117a03b4f21ab3290a15680de66, user.id=1da355c8be2d496bb08feb0fe74e04b9) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:47.663160 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c213d1-df7d-4a0d-a27e-f1f6a1a06af2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:47.664568 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c213d1-df7d-4a0d-a27e-f1f6a1a06af2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:47.670108 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 404/814] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:14:47 2022] GET /identity/v3/role_assignments?user.id=1da355c8be2d496bb08feb0fe74e04b9&scope.project.id=d18a5117a03b4f21ab3290a15680de66&role.id=f886b8e8889549e896a8fb3426cce9e1 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:47.670460 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e79ee-ef53-47c9-9478-8312034663a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:47.670926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c213d1-df7d-4a0d-a27e-f1f6a1a06af2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:47.671101 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e79ee-ef53-47c9-9478-8312034663a3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:47.671380 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e79ee-ef53-47c9-9478-8312034663a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:47.671649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367e79ee-ef53-47c9-9478-8312034663a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:47.672305 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e79ee-ef53-47c9-9478-8312034663a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:47.672603 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e79ee-ef53-47c9-9478-8312034663a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:47.674047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367e79ee-ef53-47c9-9478-8312034663a3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:47.689359 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 411/815] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Sep 2 09:14:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:47.689359 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e79ee-ef53-47c9-9478-8312034663a3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:47.694296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:47.710476 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:47.793897 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c213d1-df7d-4a0d-a27e-f1f6a1a06af2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:47.801347 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367e79ee-ef53-47c9-9478-8312034663a3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:47.834636 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282d846-098f-4b98-90b7-2dd1f5e733ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:47.835437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282d846-098f-4b98-90b7-2dd1f5e733ca None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:47.835596 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282d846-098f-4b98-90b7-2dd1f5e733ca None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:47.836722 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282d846-098f-4b98-90b7-2dd1f5e733ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:47.839750 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718518d-fe28-405a-8b29-4da70d278332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:47.839750 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718518d-fe28-405a-8b29-4da70d278332 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:47.839966 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718518d-fe28-405a-8b29-4da70d278332 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:47.840209 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5718518d-fe28-405a-8b29-4da70d278332 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:47.840840 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5718518d-fe28-405a-8b29-4da70d278332 None admin] RBAC: Adding query filter params (type=network) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:47.841141 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5718518d-fe28-405a-8b29-4da70d278332 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:47.842497 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5718518d-fe28-405a-8b29-4da70d278332 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:47.850153 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 412/816] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Sep 2 09:14:47 2022] GET /identity/v3/services?type=network => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:47.851057 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718518d-fe28-405a-8b29-4da70d278332 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:47.855808 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1282d846-098f-4b98-90b7-2dd1f5e733ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:47.857271 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1282d846-098f-4b98-90b7-2dd1f5e733ca None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:47.873060 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-1282d846-098f-4b98-90b7-2dd1f5e733ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:47.873060 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:47.873060 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:47.873060 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:47.873060 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:47.873060 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:47.873060 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:47.873060 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:47.873060 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:47.873060 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:47.873060 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:47.873060 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:47.873060 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:47.873060 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:47.873060 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:47.873060 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:47.874050 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:47.874050 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:47.874050 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:47.874050 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:47.874050 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:47.874050 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:47.874050 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:47.874050 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:47.874050 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:47.874050 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:47.874050 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:47.874050 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:47.874050 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:47.874050 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:47.874050 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:47.874050 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:47.874050 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:47.874976 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:47.874976 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:47.874976 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:47.879166 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 405/817] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:47.880112 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282d846-098f-4b98-90b7-2dd1f5e733ca None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:47.939808 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:47.948259 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:47.959253 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5718518d-fe28-405a-8b29-4da70d278332 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:47.987676 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282d846-098f-4b98-90b7-2dd1f5e733ca None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:47.993106 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a45097-a243-4a58-bfca-891f3d06bb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:47.993716 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a45097-a243-4a58-bfca-891f3d06bb45 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:47.993996 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a45097-a243-4a58-bfca-891f3d06bb45 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:47.994270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a45097-a243-4a58-bfca-891f3d06bb45 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:47.994976 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a45097-a243-4a58-bfca-891f3d06bb45 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:47.996413 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a45097-a243-4a58-bfca-891f3d06bb45 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:48.022642 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647bfc7-6712-4546-a328-83f342177534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:48.023356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647bfc7-6712-4546-a328-83f342177534 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:48.023818 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647bfc7-6712-4546-a328-83f342177534 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:48.024075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647bfc7-6712-4546-a328-83f342177534 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:48.035253 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647bfc7-6712-4546-a328-83f342177534 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:48.037335 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647bfc7-6712-4546-a328-83f342177534 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:48.051464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-a647bfc7-6712-4546-a328-83f342177534 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:48.051464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:48.051464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:48.051464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:48.051464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:48.051464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:48.051464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:48.051464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:48.051464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:48.051464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:48.051464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:48.051464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:48.051464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:48.051464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:48.051464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:48.051464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:48.052780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:48.052780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:48.052780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:48.052780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:48.052780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:48.052780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:48.052780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:48.052780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:48.052780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:48.052780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:48.052780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:48.052780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:48.052780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:48.052780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:48.052780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:48.052780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:48.052780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:48.053681 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:48.053681 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:48.053681 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:48.053681 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 406/818] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:48.053904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647bfc7-6712-4546-a328-83f342177534 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:48.146826 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 413/819] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:14:47 2022] POST /identity/v3/endpoints => generated 346 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:14:48.146826 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a45097-a243-4a58-bfca-891f3d06bb45 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:48.166274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647bfc7-6712-4546-a328-83f342177534 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:48.188247 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:48.216766 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae565a0f-6736-4960-94bd-c81b163a1261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:48.217493 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae565a0f-6736-4960-94bd-c81b163a1261 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:48.217845 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae565a0f-6736-4960-94bd-c81b163a1261 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:48.218215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae565a0f-6736-4960-94bd-c81b163a1261 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:48.219040 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae565a0f-6736-4960-94bd-c81b163a1261 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:48.220580 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae565a0f-6736-4960-94bd-c81b163a1261 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:48.279445 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a45097-a243-4a58-bfca-891f3d06bb45 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:48.283608 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 407/820] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:14:48 2022] POST /identity/v3/endpoints => generated 337 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:14:48.285248 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae565a0f-6736-4960-94bd-c81b163a1261 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:48.349632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255fece1-547d-4572-b0f4-94be71e625e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:48.350697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fece1-547d-4572-b0f4-94be71e625e3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:48.351308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fece1-547d-4572-b0f4-94be71e625e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:48.351788 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fece1-547d-4572-b0f4-94be71e625e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:48.352742 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255fece1-547d-4572-b0f4-94be71e625e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:48.353355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255fece1-547d-4572-b0f4-94be71e625e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:48.355229 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255fece1-547d-4572-b0f4-94be71e625e3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:48.357588 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:48.372461 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 414/821] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:48.378949 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255fece1-547d-4572-b0f4-94be71e625e3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:48.435541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae565a0f-6736-4960-94bd-c81b163a1261 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:48.482294 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90450326-7671-44f2-8621-20b50e099173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:48.483374 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90450326-7671-44f2-8621-20b50e099173 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:48.484003 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90450326-7671-44f2-8621-20b50e099173 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:48.484498 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90450326-7671-44f2-8621-20b50e099173 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:48.485416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90450326-7671-44f2-8621-20b50e099173 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:48.485988 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90450326-7671-44f2-8621-20b50e099173 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:48.487825 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90450326-7671-44f2-8621-20b50e099173 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:48.502119 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 408/822] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:48.509627 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90450326-7671-44f2-8621-20b50e099173 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:48.511978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255fece1-547d-4572-b0f4-94be71e625e3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:48.550999 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3459f21f-f92f-4c27-a474-26bc1d1c0689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:48.551597 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459f21f-f92f-4c27-a474-26bc1d1c0689 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:48.551823 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459f21f-f92f-4c27-a474-26bc1d1c0689 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:48.552096 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459f21f-f92f-4c27-a474-26bc1d1c0689 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:48.555076 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:48.561053 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3459f21f-f92f-4c27-a474-26bc1d1c0689 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:48.562493 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3459f21f-f92f-4c27-a474-26bc1d1c0689 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:48.576749 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-3459f21f-f92f-4c27-a474-26bc1d1c0689 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:48.576749 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:48.576749 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:48.576749 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:48.576749 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:48.576749 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:48.576749 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:48.576749 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:48.576749 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:48.576749 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:48.576749 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:48.576749 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:48.576749 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:14:48.576749 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:14:48.576749 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:14:48.583463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:14:48.583463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:48.583463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:48.583463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:48.583463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:48.583463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:48.583463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:48.583463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:48.583463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:48.583463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:48.583463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:48.583463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:48.583463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:48.583463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:48.583463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:48.583463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:48.583463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:48.585116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:14:48.585116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:14:48.585116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:14:48.585116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:14:48.585116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:14:48.585116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:14:48.585116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:48.585116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:48.585116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 415/823] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:14:48.588443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3459f21f-f92f-4c27-a474-26bc1d1c0689 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:48.658947 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90450326-7671-44f2-8621-20b50e099173 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:48.670328 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:48.718835 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692dffb4-cbd3-4e6f-8be9-b1eee922118a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:48.719736 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692dffb4-cbd3-4e6f-8be9-b1eee922118a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:48.720447 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692dffb4-cbd3-4e6f-8be9-b1eee922118a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:48.720822 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692dffb4-cbd3-4e6f-8be9-b1eee922118a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:48.723324 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3459f21f-f92f-4c27-a474-26bc1d1c0689 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:48.730596 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692dffb4-cbd3-4e6f-8be9-b1eee922118a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:48.732638 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692dffb4-cbd3-4e6f-8be9-b1eee922118a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:48.747771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-692dffb4-cbd3-4e6f-8be9-b1eee922118a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 09:14:48.747771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:48.747771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:48.747771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:48.747771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:48.747771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:48.747771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:48.747771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:48.747771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:48.747771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:48.747771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:48.747771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:48.747771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:14:48.747771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:14:48.747771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:14:48.747771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:14:48.748871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:48.748871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:48.748871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:14:48.748871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:48.748871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:14:48.748871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:48.748871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:48.748871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:48.748871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:48.748871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:48.748871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:48.748871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:48.748871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:48.748871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:48.748871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:48.748871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:48.748871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:48.748871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:48.749805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:48.749805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:48.749805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:14:48.749805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:14:48.749805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:14:48.749805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:14:48.749805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:14:48.749805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:14:48.749805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 09:14:48.749805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:48.749805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 409/824] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Sep 2 09:14:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:48.755948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692dffb4-cbd3-4e6f-8be9-b1eee922118a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:48.767499 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171fb4a-bf58-4c50-b21e-47b736d28f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:48.768438 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171fb4a-bf58-4c50-b21e-47b736d28f41 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:48.769036 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171fb4a-bf58-4c50-b21e-47b736d28f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:48.769535 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171fb4a-bf58-4c50-b21e-47b736d28f41 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:48.770532 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171fb4a-bf58-4c50-b21e-47b736d28f41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:48.771661 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171fb4a-bf58-4c50-b21e-47b736d28f41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:48.773387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171fb4a-bf58-4c50-b21e-47b736d28f41 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:48.794762 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 416/825] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:14:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:14:48.796362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171fb4a-bf58-4c50-b21e-47b736d28f41 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:48.869093 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:48.883956 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692dffb4-cbd3-4e6f-8be9-b1eee922118a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:48.910731 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171fb4a-bf58-4c50-b21e-47b736d28f41 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:48.926179 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2dba6e-6ac8-439c-8a20-87680a8bbd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:48.926586 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:48.926822 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2dba6e-6ac8-439c-8a20-87680a8bbd5a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:48.927104 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2dba6e-6ac8-439c-8a20-87680a8bbd5a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:48.927408 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2dba6e-6ac8-439c-8a20-87680a8bbd5a None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:48.928498 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2dba6e-6ac8-439c-8a20-87680a8bbd5a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:48.928834 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2dba6e-6ac8-439c-8a20-87680a8bbd5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:48.930570 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2dba6e-6ac8-439c-8a20-87680a8bbd5a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:48.952709 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 410/826] 10.222.0.57 () {56 vars in 1149 bytes} [Fri Sep 2 09:14:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:48.957890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2dba6e-6ac8-439c-8a20-87680a8bbd5a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:48.960663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b99a19-df54-4627-9f53-481a2df6c357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:48.961450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b99a19-df54-4627-9f53-481a2df6c357 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:48.961728 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b99a19-df54-4627-9f53-481a2df6c357 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:48.962004 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b99a19-df54-4627-9f53-481a2df6c357 None admin] PATH_INFO: `/v3/projects/d18a5117a03b4f21ab3290a15680de66/users/30fe233a7d4c4fbea6ede233bd9ff8c8/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:48.965053 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b99a19-df54-4627-9f53-481a2df6c357 None admin] RBAC: Authorizing `identity:create_grant(project_id=d18a5117a03b4f21ab3290a15680de66, user_id=30fe233a7d4c4fbea6ede233bd9ff8c8, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:48.966492 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b99a19-df54-4627-9f53-481a2df6c357 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:49.033423 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 417/827] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Sep 2 09:14:48 2022] PUT /identity/v3/projects/d18a5117a03b4f21ab3290a15680de66/users/30fe233a7d4c4fbea6ede233bd9ff8c8/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:14:49.073081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2dba6e-6ac8-439c-8a20-87680a8bbd5a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:49.114230 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea9697-5ee9-4100-932b-7b6a085f4bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:49.115885 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea9697-5ee9-4100-932b-7b6a085f4bda None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:49.116398 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea9697-5ee9-4100-932b-7b6a085f4bda None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:49.116876 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ea9697-5ee9-4100-932b-7b6a085f4bda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:49.128944 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea9697-5ee9-4100-932b-7b6a085f4bda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:49.130847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ea9697-5ee9-4100-932b-7b6a085f4bda None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:49.139148 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:49.149223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-e7ea9697-5ee9-4100-932b-7b6a085f4bda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:49.149223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:49.149223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:49.149223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:49.149223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:49.149223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:49.149223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:49.149223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:49.149223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:49.149223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:49.149223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:49.149223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:49.149223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:49.149223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:49.149223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:49.149223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:49.157923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:49.157923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:49.157923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:49.157923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:49.157923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:49.157923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:49.157923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:49.157923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:49.157923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:49.157923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:49.157923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:49.157923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:49.157923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:49.157923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:49.157923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:49.157923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:49.157923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:49.158873 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:49.158873 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:49.158873 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:49.158873 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 411/828] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:49.159136 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b99a19-df54-4627-9f53-481a2df6c357 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:49.269675 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b99a19-df54-4627-9f53-481a2df6c357 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:49.307059 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5357a6-eb81-4bf1-a643-86c9177b1b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:49.307916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5357a6-eb81-4bf1-a643-86c9177b1b8e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:49.308362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5357a6-eb81-4bf1-a643-86c9177b1b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:49.308783 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5357a6-eb81-4bf1-a643-86c9177b1b8e None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:49.309579 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5357a6-eb81-4bf1-a643-86c9177b1b8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:49.310030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5357a6-eb81-4bf1-a643-86c9177b1b8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:49.311626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5357a6-eb81-4bf1-a643-86c9177b1b8e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:49.324431 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 418/829] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:49.330309 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea9697-5ee9-4100-932b-7b6a085f4bda None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:49.442230 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ea9697-5ee9-4100-932b-7b6a085f4bda None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:49.479481 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4b12e-5302-4f93-9708-7fac2a25d45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:49.480114 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4b12e-5302-4f93-9708-7fac2a25d45a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:49.480398 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4b12e-5302-4f93-9708-7fac2a25d45a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:49.480668 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4b12e-5302-4f93-9708-7fac2a25d45a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:49.487717 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4b12e-5302-4f93-9708-7fac2a25d45a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:49.489214 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4b12e-5302-4f93-9708-7fac2a25d45a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:49.501083 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-13a4b12e-5302-4f93-9708-7fac2a25d45a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:49.501083 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:49.501083 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:49.501083 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:49.501083 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:49.501083 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:49.501083 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:49.501083 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:49.501083 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:49.501083 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:49.501083 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:49.501083 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:49.501083 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:14:49.501083 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:14:49.501083 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:14:49.502068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:14:49.502068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:49.502068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:49.502068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:49.502068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:49.502068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:49.502068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:49.502068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:49.502068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:49.502068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:49.502068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:49.502068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:49.502068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:49.502068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:49.502068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:49.502068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:49.502068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:49.503021 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:14:49.503021 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:14:49.503021 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:14:49.503021 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:14:49.503021 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:14:49.503021 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:14:49.503021 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:49.503021 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:49.503021 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 412/830] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:49.509689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5357a6-eb81-4bf1-a643-86c9177b1b8e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:49.546648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:49.619776 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5357a6-eb81-4bf1-a643-86c9177b1b8e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:49.660542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ce014-2972-4e37-9a6a-f5ef7270ca4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:49.661403 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ce014-2972-4e37-9a6a-f5ef7270ca4d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:49.661889 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ce014-2972-4e37-9a6a-f5ef7270ca4d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:49.662313 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ce014-2972-4e37-9a6a-f5ef7270ca4d None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:49.663184 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ce014-2972-4e37-9a6a-f5ef7270ca4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:49.663639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ce014-2972-4e37-9a6a-f5ef7270ca4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:49.665201 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ce014-2972-4e37-9a6a-f5ef7270ca4d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:49.678392 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 419/831] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:14:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:49.683866 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78398e-1ffd-4269-afe7-1afcbf40583e None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:49.684320 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78398e-1ffd-4269-afe7-1afcbf40583e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:49.684412 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f78398e-1ffd-4269-afe7-1afcbf40583e None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:49.685409 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ce014-2972-4e37-9a6a-f5ef7270ca4d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:49.685848 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 413/832] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:49.690464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9316b0-2e9f-4c62-8b4f-780f9dafe7ab None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:49.690763 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9316b0-2e9f-4c62-8b4f-780f9dafe7ab None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:49.691044 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9316b0-2e9f-4c62-8b4f-780f9dafe7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:49.714042 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ec9316b0-2e9f-4c62-8b4f-780f9dafe7ab None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:49.715988 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ec9316b0-2e9f-4c62-8b4f-780f9dafe7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:49.788418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:49.796079 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ce014-2972-4e37-9a6a-f5ef7270ca4d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:49.836903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e21222-da75-4aaf-a32f-4649128a2303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:49.837583 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e21222-da75-4aaf-a32f-4649128a2303 None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:49.837860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e21222-da75-4aaf-a32f-4649128a2303 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:49.838133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e21222-da75-4aaf-a32f-4649128a2303 None admin] PATH_INFO: `/v3/projects/d18a5117a03b4f21ab3290a15680de66/users/5233689a53364258b86ad2fa012d8db1/roles/faf1138f11d34abe8a862f1a1f7c140a` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:49.861513 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e21222-da75-4aaf-a32f-4649128a2303 None admin] RBAC: Authorizing `identity:create_grant(project_id=d18a5117a03b4f21ab3290a15680de66, user_id=5233689a53364258b86ad2fa012d8db1, role_id=faf1138f11d34abe8a862f1a1f7c140a)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:49.863014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e21222-da75-4aaf-a32f-4649128a2303 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:49.904823 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:49.919269 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 420/833] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Sep 2 09:14:49 2022] PUT /identity/v3/projects/d18a5117a03b4f21ab3290a15680de66/users/5233689a53364258b86ad2fa012d8db1/roles/faf1138f11d34abe8a862f1a1f7c140a => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:14:49.931022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 414/834] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:49 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 09:14:50.201862 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e21222-da75-4aaf-a32f-4649128a2303 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:50.266549 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39c1d1-2768-4d25-80b7-23332f00d8ec None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:50.267017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39c1d1-2768-4d25-80b7-23332f00d8ec None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:50.267212 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39c1d1-2768-4d25-80b7-23332f00d8ec None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:50.269935 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 415/835] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:50.274019 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4d4e0-5929-4b29-8583-5ef2f9306742 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:50.274300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4d4e0-5929-4b29-8583-5ef2f9306742 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:50.274572 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4d4e0-5929-4b29-8583-5ef2f9306742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:50.294164 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d0e4d4e0-5929-4b29-8583-5ef2f9306742 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:50.295376 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d0e4d4e0-5929-4b29-8583-5ef2f9306742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:50.325417 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e21222-da75-4aaf-a32f-4649128a2303 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:50.364279 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c1465-7bc5-49b2-88a1-09bf55150c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:50.365469 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c1465-7bc5-49b2-88a1-09bf55150c50 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:50.365771 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c1465-7bc5-49b2-88a1-09bf55150c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:50.366125 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c1465-7bc5-49b2-88a1-09bf55150c50 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:50.366729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c1465-7bc5-49b2-88a1-09bf55150c50 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:50.368289 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c1465-7bc5-49b2-88a1-09bf55150c50 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:50.375801 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 416/836] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:50 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 09:14:50.381386 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-fe0c1465-7bc5-49b2-88a1-09bf55150c50 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 02 09:14:50.381386 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:50.381386 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 09:14:50.381386 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 09:14:50.381386 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 09:14:50.381386 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 09:14:50.381386 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 09:14:50.381386 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:14:50.381386 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 02 09:14:50.381386 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:50.381386 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:14:50.381386 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:50.381386 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:50.381386 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:50.381386 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:50.381386 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:50.382340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:50.382340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:50.382340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:50.382340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:50.382340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:50.382340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:50.382340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:50.382340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 09:14:50.382340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 09:14:50.382340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 09:14:50.382340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 09:14:50.382340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:50.382340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:50.382340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:50.382340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:50.382340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:50.382340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:50.383312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:50.383312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:50.383312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:50.383312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:50.383312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:50.383312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:50.383312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:50.383312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:50.383312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:50.383312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:50.383312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 09:14:50.383312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:14:50.383312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 02 09:14:50.383312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:50.383312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 421/837] 10.222.0.57 () {56 vars in 1132 bytes} [Fri Sep 2 09:14:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:50.387202 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4d4e0-5929-4b29-8583-5ef2f9306742 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:50.487204 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4d4e0-5929-4b29-8583-5ef2f9306742 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:50.519135 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:50.523488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b71418-5a62-4717-ab76-a5a7fdd9bb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:50.523949 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b71418-5a62-4717-ab76-a5a7fdd9bb4a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:50.524512 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b71418-5a62-4717-ab76-a5a7fdd9bb4a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:50.524852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b71418-5a62-4717-ab76-a5a7fdd9bb4a None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:50.525481 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b71418-5a62-4717-ab76-a5a7fdd9bb4a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:50.525779 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b71418-5a62-4717-ab76-a5a7fdd9bb4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:50.527151 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b71418-5a62-4717-ab76-a5a7fdd9bb4a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:50.531986 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 417/838] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Sep 2 09:14:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:50.539224 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c1465-7bc5-49b2-88a1-09bf55150c50 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:50.549716 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:50.642834 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c1465-7bc5-49b2-88a1-09bf55150c50 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:50.656414 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b71418-5a62-4717-ab76-a5a7fdd9bb4a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:50.686001 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01b91f-d778-4f7c-9b90-07c3545c4c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:50.686781 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01b91f-d778-4f7c-9b90-07c3545c4c4f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:50.687194 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01b91f-d778-4f7c-9b90-07c3545c4c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:50.687470 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01b91f-d778-4f7c-9b90-07c3545c4c4f None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:50.688103 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d01b91f-d778-4f7c-9b90-07c3545c4c4f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:50.688402 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d01b91f-d778-4f7c-9b90-07c3545c4c4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:50.690232 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d01b91f-d778-4f7c-9b90-07c3545c4c4f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:50.697157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 422/839] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Sep 2 09:14:50 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:50.774807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b71418-5a62-4717-ab76-a5a7fdd9bb4a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:50.811871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d2a13-198b-4a85-8ec0-b67ac93d5ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:50.812517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d2a13-198b-4a85-8ec0-b67ac93d5ffd None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:50.812795 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d2a13-198b-4a85-8ec0-b67ac93d5ffd None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:50.813066 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d2a13-198b-4a85-8ec0-b67ac93d5ffd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:50.813647 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d2a13-198b-4a85-8ec0-b67ac93d5ffd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:50.815029 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d2a13-198b-4a85-8ec0-b67ac93d5ffd None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:50.825425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-710d2a13-198b-4a85-8ec0-b67ac93d5ffd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 02 09:14:50.825425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:50.825425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 09:14:50.825425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 09:14:50.825425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 09:14:50.825425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 09:14:50.825425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 09:14:50.825425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:14:50.825425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 02 09:14:50.825425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:50.825425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:14:50.825425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:50.825425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:50.825425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:50.825425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:50.825425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:50.825425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:50.826514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:50.826514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:50.826514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:50.826514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:50.826514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:50.826514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:50.826514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 09:14:50.826514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 09:14:50.826514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 09:14:50.826514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 09:14:50.826514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:50.826514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:50.826514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:50.826514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:50.826514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:50.826514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:50.827797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:50.827797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:50.827797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:50.827797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:50.827797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:50.827797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:50.827797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:50.827797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:50.827797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:50.827797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:50.827797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 09:14:50.827797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:14:50.827797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 02 09:14:50.827797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:50.827797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 418/840] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Sep 2 09:14:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:14:50.832792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01b91f-d778-4f7c-9b90-07c3545c4c4f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:50.918777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:50.950575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01b91f-d778-4f7c-9b90-07c3545c4c4f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:50.985928 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fe5e39-a672-492e-8dc8-943f96a46a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:50.986909 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe5e39-a672-492e-8dc8-943f96a46a9f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:50.987402 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe5e39-a672-492e-8dc8-943f96a46a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:50.987881 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fe5e39-a672-492e-8dc8-943f96a46a9f None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:50.988709 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe5e39-a672-492e-8dc8-943f96a46a9f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:50.989298 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe5e39-a672-492e-8dc8-943f96a46a9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:50.991395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fe5e39-a672-492e-8dc8-943f96a46a9f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:50.996662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 423/841] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Sep 2 09:14:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:51.002381 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d2a13-198b-4a85-8ec0-b67ac93d5ffd None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:51.054982 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:51.090942 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c6c6d-2c77-4fb6-9f52-788728ae180b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:51.091558 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c6c6d-2c77-4fb6-9f52-788728ae180b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:51.092026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c6c6d-2c77-4fb6-9f52-788728ae180b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:51.094246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 424/842] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:14:51.100162 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b60256-0618-4968-a1f0-cda556062f91 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:51.100446 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b60256-0618-4968-a1f0-cda556062f91 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:51.100730 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b60256-0618-4968-a1f0-cda556062f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:51.119271 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d2a13-198b-4a85-8ec0-b67ac93d5ffd None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:51.119721 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d2b60256-0618-4968-a1f0-cda556062f91 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:51.120917 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d2b60256-0618-4968-a1f0-cda556062f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:51.155596 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb6dce-076b-4dc2-a0fa-aa4e217ba69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:51.156641 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb6dce-076b-4dc2-a0fa-aa4e217ba69f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:51.157117 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb6dce-076b-4dc2-a0fa-aa4e217ba69f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:51.157709 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb6dce-076b-4dc2-a0fa-aa4e217ba69f None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:51.158605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadb6dce-076b-4dc2-a0fa-aa4e217ba69f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:51.159129 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadb6dce-076b-4dc2-a0fa-aa4e217ba69f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:51.160669 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadb6dce-076b-4dc2-a0fa-aa4e217ba69f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:51.173910 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 419/843] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Sep 2 09:14:51 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:51.193571 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 425/844] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:14:51.464711 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb6dce-076b-4dc2-a0fa-aa4e217ba69f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:51.569736 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadb6dce-076b-4dc2-a0fa-aa4e217ba69f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:51.581857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:51.605506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33380822-f190-408d-a40c-db6d19b612e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:51.606147 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33380822-f190-408d-a40c-db6d19b612e4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:51.606481 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33380822-f190-408d-a40c-db6d19b612e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:51.606777 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33380822-f190-408d-a40c-db6d19b612e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:51.617854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33380822-f190-408d-a40c-db6d19b612e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:51.622733 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-33380822-f190-408d-a40c-db6d19b612e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:14:51.622733 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:51.622733 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:51.622733 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:51.622733 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:51.622733 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:51.622733 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:51.622733 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:51.622733 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:51.622733 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:51.622733 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:51.622733 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:51.622733 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:51.622733 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:51.622733 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:51.622733 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:51.623719 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:51.623719 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:51.623719 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:51.623719 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:51.623719 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:51.623719 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:51.623719 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:51.623719 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:51.623719 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:51.623719 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:51.623719 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:51.623719 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:51.623719 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:51.623719 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:51.623719 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:51.623719 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:51.624795 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:51.624795 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:51.624795 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:51.624795 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:51.624795 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:51.624795 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:51.624795 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:51.624795 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:51.624795 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:51.624795 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:51.624795 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:14:51.624795 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:51.624795 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 420/845] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:14:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:51.629910 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b60256-0618-4968-a1f0-cda556062f91 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:51.695041 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:51.730718 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b60256-0618-4968-a1f0-cda556062f91 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:51.764720 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b2e67-01db-468b-b697-0ed20c68aa25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:51.765340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b2e67-01db-468b-b697-0ed20c68aa25 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:51.765617 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b2e67-01db-468b-b697-0ed20c68aa25 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:51.765884 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b2e67-01db-468b-b697-0ed20c68aa25 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:51.766543 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31b2e67-01db-468b-b697-0ed20c68aa25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:51.766878 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31b2e67-01db-468b-b697-0ed20c68aa25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:51.768230 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31b2e67-01db-468b-b697-0ed20c68aa25 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:51.777355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 426/846] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:14:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:51.782815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33380822-f190-408d-a40c-db6d19b612e4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:51.864602 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674544e5-d2b5-4305-ba39-fb4bd2ae8cf2 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:51.865128 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674544e5-d2b5-4305-ba39-fb4bd2ae8cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:51.865128 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674544e5-d2b5-4305-ba39-fb4bd2ae8cf2 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:51.866755 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 427/847] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:14:51.871706 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db96d5-2bad-48d7-b3d6-fefec5e21abb None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:51.871992 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db96d5-2bad-48d7-b3d6-fefec5e21abb None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:51.872270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db96d5-2bad-48d7-b3d6-fefec5e21abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:51.893258 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-12db96d5-2bad-48d7-b3d6-fefec5e21abb None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:51.894502 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-12db96d5-2bad-48d7-b3d6-fefec5e21abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:51.901848 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33380822-f190-408d-a40c-db6d19b612e4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:51.939602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf52b74-38a0-4b91-b93a-0fe12f2d1f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:51.939951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf52b74-38a0-4b91-b93a-0fe12f2d1f8a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:51.940232 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf52b74-38a0-4b91-b93a-0fe12f2d1f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:51.940503 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf52b74-38a0-4b91-b93a-0fe12f2d1f8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:51.952309 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf52b74-38a0-4b91-b93a-0fe12f2d1f8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:51.953747 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf52b74-38a0-4b91-b93a-0fe12f2d1f8a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:51.968896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-ccf52b74-38a0-4b91-b93a-0fe12f2d1f8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:51.968896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:51.968896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:51.968896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:51.968896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:51.968896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:51.968896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:51.968896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:51.968896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:51.968896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:51.968896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:51.968896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:51.968896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:51.968896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:51.968896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:51.968896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:51.969923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:51.969923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:51.969923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:51.969923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:51.969923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:51.969923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:51.969923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:51.969923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:51.969923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:51.969923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:51.969923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:51.969923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:51.969923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:51.969923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:51.969923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:51.969923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:51.969923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:51.971247 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:51.971247 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:51.971247 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:51.971247 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 421/848] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:51.978836 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf52b74-38a0-4b91-b93a-0fe12f2d1f8a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:51.979081 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 428/849] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 09:14:52.002530 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:52.078427 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf52b74-38a0-4b91-b93a-0fe12f2d1f8a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:52.112164 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e9ca8-22b5-449c-b2da-084e2e20d734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:52.112766 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e9ca8-22b5-449c-b2da-084e2e20d734 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:52.113041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e9ca8-22b5-449c-b2da-084e2e20d734 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:52.113308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e9ca8-22b5-449c-b2da-084e2e20d734 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:52.113933 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e9ca8-22b5-449c-b2da-084e2e20d734 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:52.114231 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e9ca8-22b5-449c-b2da-084e2e20d734 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:52.115620 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e9ca8-22b5-449c-b2da-084e2e20d734 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:52.132330 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 422/850] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:52.137555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db96d5-2bad-48d7-b3d6-fefec5e21abb None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:52.203370 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:52.246146 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db96d5-2bad-48d7-b3d6-fefec5e21abb None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:52.253319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e9ca8-22b5-449c-b2da-084e2e20d734 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:52.285874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd75e8-2509-4b1e-b30b-f46cf2e38273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:52.286980 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd75e8-2509-4b1e-b30b-f46cf2e38273 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:52.287482 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd75e8-2509-4b1e-b30b-f46cf2e38273 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:52.287943 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd75e8-2509-4b1e-b30b-f46cf2e38273 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:52.295645 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd75e8-2509-4b1e-b30b-f46cf2e38273 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:52.297348 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dd75e8-2509-4b1e-b30b-f46cf2e38273 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:52.310480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-f2dd75e8-2509-4b1e-b30b-f46cf2e38273 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 02 09:14:52.310480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:52.310480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:52.310480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:52.310480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:52.310480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:52.310480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:52.310480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:52.310480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:52.310480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:52.310480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:52.310480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:52.310480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:14:52.310480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:14:52.310480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:14:52.310480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:14:52.311500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:52.311500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:52.311500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:14:52.311500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:52.311500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:14:52.311500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:52.311500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:52.311500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:52.311500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:52.311500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:52.311500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:52.311500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:52.311500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:52.311500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:52.311500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:52.311500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:52.311500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:52.311500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:52.312436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:52.312436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:52.312436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:14:52.312436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:14:52.312436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:14:52.312436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:14:52.312436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:14:52.312436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:14:52.312436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 02 09:14:52.312436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:52.315327 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 429/851] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:14:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:52.322615 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd75e8-2509-4b1e-b30b-f46cf2e38273 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:52.353449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:52.372567 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e9ca8-22b5-449c-b2da-084e2e20d734 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:52.412015 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25db45-e7b5-47ec-a646-092ab601149b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:52.413159 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25db45-e7b5-47ec-a646-092ab601149b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:52.413447 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25db45-e7b5-47ec-a646-092ab601149b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:52.413724 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25db45-e7b5-47ec-a646-092ab601149b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:52.426644 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25db45-e7b5-47ec-a646-092ab601149b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:52.433591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-3d25db45-e7b5-47ec-a646-092ab601149b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:52.433591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:52.433591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:52.433591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:52.433591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:52.433591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:52.433591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:52.433591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:52.433591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:52.433591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:52.433591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:52.433591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:52.433591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:52.433591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:52.433591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:52.433591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:52.434514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:52.434514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:52.434514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:52.434514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:52.434514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:52.434514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:52.434514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:52.434514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:52.434514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:52.434514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:52.434514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:52.434514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:52.434514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:52.434514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:52.434514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:52.434514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:52.435478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:52.435478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:52.435478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:52.435478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:52.435478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:52.435478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:52.435478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:52.435478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:52.435478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:52.435478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:52.435478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:14:52.435478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:52.435478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 423/852] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Sep 2 09:14:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:52.440430 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25db45-e7b5-47ec-a646-092ab601149b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:52.454835 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd75e8-2509-4b1e-b30b-f46cf2e38273 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:52.472366 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:52.491833 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8d878-b080-41fd-8529-747c278abfa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:52.492061 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8d878-b080-41fd-8529-747c278abfa5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:52.492324 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8d878-b080-41fd-8529-747c278abfa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:52.492618 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8d878-b080-41fd-8529-747c278abfa5 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:52.493842 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8d878-b080-41fd-8529-747c278abfa5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:52.494167 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8d878-b080-41fd-8529-747c278abfa5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:52.495919 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8d878-b080-41fd-8529-747c278abfa5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:52.524515 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 430/853] 10.222.0.57 () {56 vars in 1135 bytes} [Fri Sep 2 09:14:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:52.527745 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8d878-b080-41fd-8529-747c278abfa5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:52.559964 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25db45-e7b5-47ec-a646-092ab601149b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:52.603346 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944ae87-df38-4911-9694-9c9945f14750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:52.603982 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944ae87-df38-4911-9694-9c9945f14750 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:52.604264 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944ae87-df38-4911-9694-9c9945f14750 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:52.604545 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944ae87-df38-4911-9694-9c9945f14750 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:52.605382 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a944ae87-df38-4911-9694-9c9945f14750 None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:52.605682 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a944ae87-df38-4911-9694-9c9945f14750 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:52.607672 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a944ae87-df38-4911-9694-9c9945f14750 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:52.622787 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 424/854] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Sep 2 09:14:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:52.624861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944ae87-df38-4911-9694-9c9945f14750 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:52.629437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:52.654402 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8d878-b080-41fd-8529-747c278abfa5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:52.700344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc530a9d-2ecb-405e-ab98-79357e65cdcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:52.701387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc530a9d-2ecb-405e-ab98-79357e65cdcc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:52.701671 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc530a9d-2ecb-405e-ab98-79357e65cdcc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:52.701946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc530a9d-2ecb-405e-ab98-79357e65cdcc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:52.750144 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944ae87-df38-4911-9694-9c9945f14750 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:52.761044 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:52.785070 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc530a9d-2ecb-405e-ab98-79357e65cdcc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:52.787065 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc530a9d-2ecb-405e-ab98-79357e65cdcc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:52.791710 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad5c8f8-c7f0-4498-9eb0-b9b1a061a938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:52.792392 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5c8f8-c7f0-4498-9eb0-b9b1a061a938 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:52.792667 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5c8f8-c7f0-4498-9eb0-b9b1a061a938 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:52.792933 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5c8f8-c7f0-4498-9eb0-b9b1a061a938 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:52.804194 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad5c8f8-c7f0-4498-9eb0-b9b1a061a938 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:52.805605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad5c8f8-c7f0-4498-9eb0-b9b1a061a938 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:52.819831 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-dad5c8f8-c7f0-4498-9eb0-b9b1a061a938 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:52.819831 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:52.819831 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:52.819831 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:52.819831 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:52.819831 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:52.819831 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:52.819831 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:52.819831 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:52.819831 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:52.819831 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:52.819831 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:52.819831 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:52.819831 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:52.819831 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:52.819831 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:52.820807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:52.820807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:52.820807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:52.820807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:52.820807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:52.820807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:52.820807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:52.820807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:52.820807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:52.820807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:52.820807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:52.820807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:52.820807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:52.820807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:52.820807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:52.820807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:52.820807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:52.821964 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:52.821964 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:52.821964 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:52.825706 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 425/855] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:14:52.827098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67229e-6d50-4268-8c2a-bc7e24286e9b None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:52.827380 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67229e-6d50-4268-8c2a-bc7e24286e9b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:52.827648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df67229e-6d50-4268-8c2a-bc7e24286e9b None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:52.829028 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 426/856] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:52.829645 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67229e-6d50-4268-8c2a-bc7e24286e9b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:52.890309 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:52.900801 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 431/857] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:14:52 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 09:14:52.900801 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa92dbd-9375-44b4-b0fb-383057e3288c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:52.900801 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa92dbd-9375-44b4-b0fb-383057e3288c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:52.900801 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa92dbd-9375-44b4-b0fb-383057e3288c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:52.921265 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8fa92dbd-9375-44b4-b0fb-383057e3288c None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:52.922454 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8fa92dbd-9375-44b4-b0fb-383057e3288c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:52.943696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df67229e-6d50-4268-8c2a-bc7e24286e9b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:52.979632 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffe581-e720-4502-a124-912425db79b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:52.980243 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffe581-e720-4502-a124-912425db79b1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:52.980546 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffe581-e720-4502-a124-912425db79b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:52.980785 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffe581-e720-4502-a124-912425db79b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:52.981423 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeffe581-e720-4502-a124-912425db79b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:52.981724 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeffe581-e720-4502-a124-912425db79b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:52.983093 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeffe581-e720-4502-a124-912425db79b1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:52.995228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 432/858] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:14:52.995228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa92dbd-9375-44b4-b0fb-383057e3288c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:53.004265 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 427/859] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:53.005206 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:53.007914 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffe581-e720-4502-a124-912425db79b1 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:53.112088 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffe581-e720-4502-a124-912425db79b1 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:53.116887 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa92dbd-9375-44b4-b0fb-383057e3288c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:53.130908 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:53.151144 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6453059-24b3-447c-b040-a3f137e3bcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:53.151913 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6453059-24b3-447c-b040-a3f137e3bcec None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:53.152193 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6453059-24b3-447c-b040-a3f137e3bcec None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:53.152419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9538e73-6980-4110-b51d-aa9ab566d291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:53.152535 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6453059-24b3-447c-b040-a3f137e3bcec None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:53.153378 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9538e73-6980-4110-b51d-aa9ab566d291 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:53.153831 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9538e73-6980-4110-b51d-aa9ab566d291 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:53.154272 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9538e73-6980-4110-b51d-aa9ab566d291 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:53.160235 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6453059-24b3-447c-b040-a3f137e3bcec None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:53.162249 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6453059-24b3-447c-b040-a3f137e3bcec None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:53.165674 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9538e73-6980-4110-b51d-aa9ab566d291 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:53.167115 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9538e73-6980-4110-b51d-aa9ab566d291 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:53.175734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d6453059-24b3-447c-b040-a3f137e3bcec None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 09:14:53.175734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:53.175734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:53.175734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:53.175734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:53.175734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:53.175734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:53.175734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:53.175734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:53.175734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:53.175734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:53.175734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:53.175734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:14:53.175734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:14:53.175734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:14:53.175734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:14:53.176685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:53.176685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:53.176685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:14:53.176685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:53.176685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:14:53.176685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:53.176685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:53.176685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:53.176685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:53.176685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:53.176685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:53.176685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:53.176685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:53.176685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:53.176685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:53.176685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:53.176685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:53.176685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:53.177662 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:53.177662 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:53.177662 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:14:53.177662 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:14:53.177662 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:14:53.177662 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:14:53.177662 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:14:53.177662 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:14:53.177662 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 09:14:53.177662 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:53.177662 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 428/860] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Sep 2 09:14:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:14:53.178792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415d8de-68e9-4dbd-a8bc-49cb8c8ff10e None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:53.178908 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415d8de-68e9-4dbd-a8bc-49cb8c8ff10e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:53.179167 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415d8de-68e9-4dbd-a8bc-49cb8c8ff10e None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:53.180531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 429/861] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:53.184710 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415d8de-68e9-4dbd-a8bc-49cb8c8ff10e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:53.186076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-e9538e73-6980-4110-b51d-aa9ab566d291 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:53.186076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:53.186076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:53.186076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:53.186076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:53.186076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:53.186076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:53.186076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:53.186076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:53.186076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:53.186076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:53.186076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:53.186076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:53.186076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:53.186076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:53.186076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:53.187216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:53.187216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:53.187216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:53.187216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:53.187216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:53.187216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:53.187216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:53.187216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:53.187216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:53.187216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:53.187216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:53.187216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:53.187216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:53.187216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:53.187216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:53.187216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:53.187216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:53.188108 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:53.188108 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:53.188108 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:53.188108 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 433/862] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:53.189278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca71657-72dc-47fc-b046-a92f806f490a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:53.189278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca71657-72dc-47fc-b046-a92f806f490a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:53.189278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca71657-72dc-47fc-b046-a92f806f490a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:53.206806 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-fca71657-72dc-47fc-b046-a92f806f490a None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:53.207981 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fca71657-72dc-47fc-b046-a92f806f490a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:53.262850 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:53.280153 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 434/863] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:14:53.280153 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca71657-72dc-47fc-b046-a92f806f490a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:53.303848 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415d8de-68e9-4dbd-a8bc-49cb8c8ff10e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:53.343906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa9139-2daf-4cb8-a6ca-3e5dd14f2d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:53.344717 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa9139-2daf-4cb8-a6ca-3e5dd14f2d00 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:53.345143 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa9139-2daf-4cb8-a6ca-3e5dd14f2d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:53.345667 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa9139-2daf-4cb8-a6ca-3e5dd14f2d00 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:53.346543 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa9139-2daf-4cb8-a6ca-3e5dd14f2d00 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:53.347019 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa9139-2daf-4cb8-a6ca-3e5dd14f2d00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:53.348564 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa9139-2daf-4cb8-a6ca-3e5dd14f2d00 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:53.366572 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 430/864] 10.222.0.57 () {56 vars in 1149 bytes} [Fri Sep 2 09:14:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:53.367626 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa9139-2daf-4cb8-a6ca-3e5dd14f2d00 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:53.385692 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca71657-72dc-47fc-b046-a92f806f490a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:53.402592 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:53.420941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc16335-c7c6-47fa-905e-c461a4dc77ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:53.421559 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc16335-c7c6-47fa-905e-c461a4dc77ad None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:53.421835 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc16335-c7c6-47fa-905e-c461a4dc77ad None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:53.422104 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc16335-c7c6-47fa-905e-c461a4dc77ad None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:53.422765 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc16335-c7c6-47fa-905e-c461a4dc77ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:53.423068 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc16335-c7c6-47fa-905e-c461a4dc77ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:53.424422 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc16335-c7c6-47fa-905e-c461a4dc77ad None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:53.440458 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 435/865] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:53.440458 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc16335-c7c6-47fa-905e-c461a4dc77ad None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:53.476224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa9139-2daf-4cb8-a6ca-3e5dd14f2d00 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:53.486939 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:53.513621 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855b8d2-4d13-48aa-8cfe-c6e420f9fff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:53.514230 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855b8d2-4d13-48aa-8cfe-c6e420f9fff6 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:53.514504 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855b8d2-4d13-48aa-8cfe-c6e420f9fff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:53.514795 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855b8d2-4d13-48aa-8cfe-c6e420f9fff6 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:53.515435 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7855b8d2-4d13-48aa-8cfe-c6e420f9fff6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:53.516783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7855b8d2-4d13-48aa-8cfe-c6e420f9fff6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:53.539548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc16335-c7c6-47fa-905e-c461a4dc77ad None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:53.572883 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b3340-232f-4dbb-9eb8-5f626daaad36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:53.573496 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b3340-232f-4dbb-9eb8-5f626daaad36 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:53.573781 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b3340-232f-4dbb-9eb8-5f626daaad36 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:53.574057 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b3340-232f-4dbb-9eb8-5f626daaad36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:53.582799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 431/866] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Sep 2 09:14:53 2022] POST /identity/v3/services => generated 269 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:14:53.584538 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855b8d2-4d13-48aa-8cfe-c6e420f9fff6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:53.656068 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b3340-232f-4dbb-9eb8-5f626daaad36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:53.657455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b3340-232f-4dbb-9eb8-5f626daaad36 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:53.699502 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:53.714802 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:53.719309 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855b8d2-4d13-48aa-8cfe-c6e420f9fff6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:53.759634 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894a328-a299-4a4a-b778-7749670ada86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:53.760322 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894a328-a299-4a4a-b778-7749670ada86 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:53.760607 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894a328-a299-4a4a-b778-7749670ada86 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:53.760874 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e894a328-a299-4a4a-b778-7749670ada86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:53.769513 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 436/867] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:14:53 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 09:14:53.770337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b3340-232f-4dbb-9eb8-5f626daaad36 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:53.771620 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894a328-a299-4a4a-b778-7749670ada86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:53.773116 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e894a328-a299-4a4a-b778-7749670ada86 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:53.785215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-e894a328-a299-4a4a-b778-7749670ada86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:53.785215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:53.785215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:53.785215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:53.785215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:53.785215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:53.785215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:53.785215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:53.785215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:53.785215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:53.785215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:53.785215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:53.785215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:14:53.785215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:14:53.785215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:14:53.786213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:14:53.786213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:53.786213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:53.786213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:53.786213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:53.786213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:53.786213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:53.786213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:53.786213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:53.786213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:53.786213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:53.786213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:53.786213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:53.786213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:53.786213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:53.786213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:53.786213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:53.787639 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:14:53.787639 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:14:53.787639 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:14:53.787639 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:14:53.787639 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:14:53.787639 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:14:53.787639 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:53.787639 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:53.787639 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 432/868] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:53.787639 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894a328-a299-4a4a-b778-7749670ada86 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:53.890136 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b3340-232f-4dbb-9eb8-5f626daaad36 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:53.897346 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e894a328-a299-4a4a-b778-7749670ada86 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:53.929289 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a6cca-cc4d-4df9-b13f-a8715ba9c25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:53.930516 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a6cca-cc4d-4df9-b13f-a8715ba9c25d None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:53.930875 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a6cca-cc4d-4df9-b13f-a8715ba9c25d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:53.931158 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a6cca-cc4d-4df9-b13f-a8715ba9c25d None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:53.931817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a6cca-cc4d-4df9-b13f-a8715ba9c25d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:53.931990 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275a0f8-60d4-4c72-b22b-05efcf929639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:53.932614 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275a0f8-60d4-4c72-b22b-05efcf929639 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:53.932896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275a0f8-60d4-4c72-b22b-05efcf929639 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:53.933172 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275a0f8-60d4-4c72-b22b-05efcf929639 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:53.933267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a6cca-cc4d-4df9-b13f-a8715ba9c25d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:53.944612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275a0f8-60d4-4c72-b22b-05efcf929639 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:53.947598 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f275a0f8-60d4-4c72-b22b-05efcf929639 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:53.960687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-f275a0f8-60d4-4c72-b22b-05efcf929639 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:53.960687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:53.960687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:53.960687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:53.960687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:53.960687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:53.960687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:53.960687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:53.960687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:53.960687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:53.960687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:53.960687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:53.960687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:53.960687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:53.960687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:53.960687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:53.961816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:53.961816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:53.961816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:53.961816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:53.961816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:53.961816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:53.961816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:53.961816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:53.961816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:53.961816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:53.961816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:53.961816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:53.961816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:53.961816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:53.961816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:53.961816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:53.961816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:53.962804 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:53.962804 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:53.962804 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:53.962804 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 433/869] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:53.962804 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275a0f8-60d4-4c72-b22b-05efcf929639 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:53.965661 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:54.018014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 437/870] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Sep 2 09:14:53 2022] POST /identity/v3/services => generated 251 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:14:54.019115 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a6cca-cc4d-4df9-b13f-a8715ba9c25d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:54.036252 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:54.096688 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275a0f8-60d4-4c72-b22b-05efcf929639 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:54.157249 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106b837-f24e-4400-9041-68fdadabedda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:54.158905 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106b837-f24e-4400-9041-68fdadabedda None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:54.159366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106b837-f24e-4400-9041-68fdadabedda None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:54.159841 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106b837-f24e-4400-9041-68fdadabedda None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:54.160537 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9106b837-f24e-4400-9041-68fdadabedda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:54.160846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9106b837-f24e-4400-9041-68fdadabedda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:54.162806 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a6cca-cc4d-4df9-b13f-a8715ba9c25d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:54.162923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9106b837-f24e-4400-9041-68fdadabedda None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:54.178741 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 434/871] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:14:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:54.184830 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106b837-f24e-4400-9041-68fdadabedda None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:54.208727 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d944ed-a712-4f8b-a75e-d68783144bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:54.209657 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d944ed-a712-4f8b-a75e-d68783144bdf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:54.210233 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d944ed-a712-4f8b-a75e-d68783144bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:54.210507 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d944ed-a712-4f8b-a75e-d68783144bdf None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:54.212708 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d944ed-a712-4f8b-a75e-d68783144bdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:54.212708 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d944ed-a712-4f8b-a75e-d68783144bdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:54.212908 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d944ed-a712-4f8b-a75e-d68783144bdf None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:54.225790 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 438/872] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:54.231548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d944ed-a712-4f8b-a75e-d68783144bdf None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:54.288739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:54.309064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:54.317169 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106b837-f24e-4400-9041-68fdadabedda None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:54.345777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d944ed-a712-4f8b-a75e-d68783144bdf None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:54.353359 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3297f02-3823-43e5-bde6-5d8837978f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:54.354441 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3297f02-3823-43e5-bde6-5d8837978f8c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:54.355187 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3297f02-3823-43e5-bde6-5d8837978f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:54.355187 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3297f02-3823-43e5-bde6-5d8837978f8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:54.356131 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3297f02-3823-43e5-bde6-5d8837978f8c None admin] RBAC: Adding query filter params (role.id=f886b8e8889549e896a8fb3426cce9e1, scope.project.id=d18a5117a03b4f21ab3290a15680de66, user.id=30fe233a7d4c4fbea6ede233bd9ff8c8) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:54.357038 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3297f02-3823-43e5-bde6-5d8837978f8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:54.358557 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3297f02-3823-43e5-bde6-5d8837978f8c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:54.370740 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 435/873] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:14:54 2022] GET /identity/v3/role_assignments?user.id=30fe233a7d4c4fbea6ede233bd9ff8c8&scope.project.id=d18a5117a03b4f21ab3290a15680de66&role.id=f886b8e8889549e896a8fb3426cce9e1 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:14:54.390933 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70adef-d400-4658-a722-b09f2df0f9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:54.391594 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70adef-d400-4658-a722-b09f2df0f9c5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:54.391877 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70adef-d400-4658-a722-b09f2df0f9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:54.392157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e70adef-d400-4658-a722-b09f2df0f9c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:54.400148 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e70adef-d400-4658-a722-b09f2df0f9c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:54.401688 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e70adef-d400-4658-a722-b09f2df0f9c5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:54.415777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-9e70adef-d400-4658-a722-b09f2df0f9c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:54.415777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:54.415777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:54.415777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:54.415777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:54.415777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:54.415777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:54.415777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:54.415777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:54.415777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:54.415777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:54.415777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:54.415777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:14:54.415777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:14:54.415777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:14:54.416717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:14:54.416717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:54.416717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:54.416717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:54.416717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:54.416717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:54.416717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:54.416717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:54.416717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:54.416717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:54.416717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:54.416717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:54.416717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:54.416717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:54.416717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:54.416717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:54.416717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:54.417812 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:14:54.417812 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:14:54.417812 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:14:54.417812 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:14:54.417812 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:14:54.417812 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:14:54.417812 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:54.417812 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:54.417812 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 439/874] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:54.422722 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3297f02-3823-43e5-bde6-5d8837978f8c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:54.534156 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3297f02-3823-43e5-bde6-5d8837978f8c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:54.558344 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:54.569841 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b87bae-b08d-4006-9b58-532ed7c41a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:54.570455 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b87bae-b08d-4006-9b58-532ed7c41a86 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:54.570746 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b87bae-b08d-4006-9b58-532ed7c41a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:54.571026 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b87bae-b08d-4006-9b58-532ed7c41a86 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:54.571969 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b87bae-b08d-4006-9b58-532ed7c41a86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:54.572272 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b87bae-b08d-4006-9b58-532ed7c41a86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:54.573687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b87bae-b08d-4006-9b58-532ed7c41a86 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:54.586386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 436/875] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:14:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:54.592806 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70adef-d400-4658-a722-b09f2df0f9c5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:54.699435 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e70adef-d400-4658-a722-b09f2df0f9c5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:54.730426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:54.739123 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58710c7-07db-4f3c-a84c-0daa16aa1609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:54.740034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58710c7-07db-4f3c-a84c-0daa16aa1609 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:54.740532 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58710c7-07db-4f3c-a84c-0daa16aa1609 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:54.741014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58710c7-07db-4f3c-a84c-0daa16aa1609 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:54.742038 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58710c7-07db-4f3c-a84c-0daa16aa1609 None admin] RBAC: Adding query filter params (role.id=faf1138f11d34abe8a862f1a1f7c140a, scope.project.id=d18a5117a03b4f21ab3290a15680de66, user.id=5233689a53364258b86ad2fa012d8db1) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:54.742557 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58710c7-07db-4f3c-a84c-0daa16aa1609 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:54.744179 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58710c7-07db-4f3c-a84c-0daa16aa1609 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:54.750927 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 440/876] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:14:54 2022] GET /identity/v3/role_assignments?user.id=5233689a53364258b86ad2fa012d8db1&scope.project.id=d18a5117a03b4f21ab3290a15680de66&role.id=faf1138f11d34abe8a862f1a1f7c140a => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:55.548771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5963db0-d884-4741-99b9-6035ce8b4708 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:55.549330 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5963db0-d884-4741-99b9-6035ce8b4708 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:55.549330 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5963db0-d884-4741-99b9-6035ce8b4708 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:55.551187 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 437/877] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:55.557373 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d95e9-b9b0-403c-87c2-c5d47604a0ec None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:55.557851 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d95e9-b9b0-403c-87c2-c5d47604a0ec None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:55.557955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d95e9-b9b0-403c-87c2-c5d47604a0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:55.577870 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-863d95e9-b9b0-403c-87c2-c5d47604a0ec None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:55.579081 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-863d95e9-b9b0-403c-87c2-c5d47604a0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:55.647449 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 441/878] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:55 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:14:55.916794 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5963db0-d884-4741-99b9-6035ce8b4708 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:56.029412 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5963db0-d884-4741-99b9-6035ce8b4708 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:56.070051 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35c923-17b8-48fb-8cd6-8fca914d46d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:56.070686 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35c923-17b8-48fb-8cd6-8fca914d46d1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:56.070995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35c923-17b8-48fb-8cd6-8fca914d46d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:56.071274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35c923-17b8-48fb-8cd6-8fca914d46d1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:56.072109 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35c923-17b8-48fb-8cd6-8fca914d46d1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:56.073461 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc35c923-17b8-48fb-8cd6-8fca914d46d1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:56.082506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-bc35c923-17b8-48fb-8cd6-8fca914d46d1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 02 09:14:56.082506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:56.082506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 09:14:56.082506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 09:14:56.082506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 09:14:56.082506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 09:14:56.082506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 09:14:56.082506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:14:56.082506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 02 09:14:56.082506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:56.082506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:14:56.082506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:56.082506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:56.082506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:56.082506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:56.082506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:56.084022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:56.084022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:56.084022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:56.084022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:56.084022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:56.084022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:56.084022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:56.084022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 09:14:56.084022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 09:14:56.084022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 09:14:56.084022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 09:14:56.084022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:56.084022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:56.084022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:56.084022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:56.084022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:56.084022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:56.084951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:56.084951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:56.084951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:56.084951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:56.084951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:56.084951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:56.084951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:56.084951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:56.084951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:56.084951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:56.084951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 09:14:56.084951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:14:56.084951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 02 09:14:56.084951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:56.084951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 438/879] 10.222.0.57 () {56 vars in 1132 bytes} [Fri Sep 2 09:14:55 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:56.090521 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d95e9-b9b0-403c-87c2-c5d47604a0ec None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:56.103239 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd32d3-adcc-419c-b2e7-f5c09b0e468a None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:56.103454 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd32d3-adcc-419c-b2e7-f5c09b0e468a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:56.103733 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd32d3-adcc-419c-b2e7-f5c09b0e468a None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:56.105358 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 439/880] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:14:56.110310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4234114-1929-42bd-a59c-4d6bc0cbff36 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:56.110599 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4234114-1929-42bd-a59c-4d6bc0cbff36 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:56.110914 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4234114-1929-42bd-a59c-4d6bc0cbff36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:56.129944 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e4234114-1929-42bd-a59c-4d6bc0cbff36 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:56.131188 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e4234114-1929-42bd-a59c-4d6bc0cbff36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:56.178942 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:56.203900 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d95e9-b9b0-403c-87c2-c5d47604a0ec None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:56.206785 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 440/881] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:56 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 09:14:56.239058 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:56.243773 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceac198-673e-4dee-8857-35a2e526e3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:56.244442 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceac198-673e-4dee-8857-35a2e526e3b0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:56.244726 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceac198-673e-4dee-8857-35a2e526e3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:56.244996 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceac198-673e-4dee-8857-35a2e526e3b0 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:56.245620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceac198-673e-4dee-8857-35a2e526e3b0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:56.245923 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceac198-673e-4dee-8857-35a2e526e3b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:56.247382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceac198-673e-4dee-8857-35a2e526e3b0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:56.252420 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 442/882] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Sep 2 09:14:56 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:56.258351 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4234114-1929-42bd-a59c-4d6bc0cbff36 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:56.371910 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4234114-1929-42bd-a59c-4d6bc0cbff36 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:56.409122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432d414-113e-45e3-b695-a008aa12103d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:56.410082 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432d414-113e-45e3-b695-a008aa12103d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:56.410577 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432d414-113e-45e3-b695-a008aa12103d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:56.411106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432d414-113e-45e3-b695-a008aa12103d None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:56.411942 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e432d414-113e-45e3-b695-a008aa12103d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:56.412472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e432d414-113e-45e3-b695-a008aa12103d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:56.414055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e432d414-113e-45e3-b695-a008aa12103d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:56.416655 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e2f29-45c3-4ff2-8d32-f1a82ba102a8 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:56.416655 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e2f29-45c3-4ff2-8d32-f1a82ba102a8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:56.416936 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e2f29-45c3-4ff2-8d32-f1a82ba102a8 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:56.418525 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 443/883] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:14:56.420479 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 441/884] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Sep 2 09:14:56 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:56.427268 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5131225-54fc-41e6-bbe0-351e5e65da53 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:56.427268 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5131225-54fc-41e6-bbe0-351e5e65da53 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:56.427268 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5131225-54fc-41e6-bbe0-351e5e65da53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:56.427559 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432d414-113e-45e3-b695-a008aa12103d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:56.441776 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c5131225-54fc-41e6-bbe0-351e5e65da53 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:56.443832 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c5131225-54fc-41e6-bbe0-351e5e65da53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:56.474343 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:56.524393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 444/885] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:56 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:14:56.525584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5131225-54fc-41e6-bbe0-351e5e65da53 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:56.547289 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432d414-113e-45e3-b695-a008aa12103d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:56.594032 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ade95-b339-407d-973d-fa7b372de6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:56.594663 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ade95-b339-407d-973d-fa7b372de6f3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:56.594968 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ade95-b339-407d-973d-fa7b372de6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:56.595474 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ade95-b339-407d-973d-fa7b372de6f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:56.596184 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ade95-b339-407d-973d-fa7b372de6f3 None admin] RBAC: Adding query filter params (interface=public, service_id=bfc863ccf942426dbd507a6356d0d604, region_id=RegionOne) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:56.596489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ade95-b339-407d-973d-fa7b372de6f3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:56.598491 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8ade95-b339-407d-973d-fa7b372de6f3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:56.605367 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 442/886] 10.222.0.57 () {56 vars in 1251 bytes} [Fri Sep 2 09:14:56 2022] GET /identity/v3/endpoints?service_id=bfc863ccf942426dbd507a6356d0d604&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:56.610306 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ade95-b339-407d-973d-fa7b372de6f3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:56.644640 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5131225-54fc-41e6-bbe0-351e5e65da53 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:56.664443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:56.684096 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22174aac-4707-4b9b-b38c-91ee6e1c7e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:56.685065 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22174aac-4707-4b9b-b38c-91ee6e1c7e14 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:56.685553 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22174aac-4707-4b9b-b38c-91ee6e1c7e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:56.686052 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22174aac-4707-4b9b-b38c-91ee6e1c7e14 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:56.686874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22174aac-4707-4b9b-b38c-91ee6e1c7e14 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:56.688543 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22174aac-4707-4b9b-b38c-91ee6e1c7e14 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:56.700310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-22174aac-4707-4b9b-b38c-91ee6e1c7e14 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 02 09:14:56.700310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:56.700310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 09:14:56.700310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 09:14:56.700310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 09:14:56.700310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 09:14:56.700310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 09:14:56.700310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:14:56.700310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 02 09:14:56.700310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:56.700310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:14:56.700310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:56.700310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:56.700310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:56.700310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:56.700310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:56.700310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:56.701302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:56.701302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:56.701302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:56.701302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:56.701302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:56.701302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:56.701302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 09:14:56.701302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 09:14:56.701302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 09:14:56.701302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 09:14:56.701302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:56.701302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:56.701302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:56.701302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:56.701302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:56.701302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:56.702132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:56.702132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:56.702132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:56.702132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:56.702132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:56.702132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:56.702132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:56.702132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:56.702132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:56.702132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:56.702132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 09:14:56.702132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:14:56.702132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 02 09:14:56.702132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:56.704892 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 445/887] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Sep 2 09:14:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:56.711283 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22174aac-4707-4b9b-b38c-91ee6e1c7e14 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:56.720861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ade95-b339-407d-973d-fa7b372de6f3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:56.723853 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:56.760874 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee133aa-06f4-4bc0-bf6e-a4ed147a4ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:56.761840 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee133aa-06f4-4bc0-bf6e-a4ed147a4ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:56.762421 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee133aa-06f4-4bc0-bf6e-a4ed147a4ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:56.762959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee133aa-06f4-4bc0-bf6e-a4ed147a4ad0 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:56.763841 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee133aa-06f4-4bc0-bf6e-a4ed147a4ad0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:56.765492 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee133aa-06f4-4bc0-bf6e-a4ed147a4ad0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:56.772747 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 443/888] 10.222.0.57 () {56 vars in 1087 bytes} [Fri Sep 2 09:14:56 2022] GET /identity/v3/services => generated 1268 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:14:56.774290 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d924b31-7b27-4cc2-adfd-19413119b4c5 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:56.774839 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d924b31-7b27-4cc2-adfd-19413119b4c5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:56.775319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d924b31-7b27-4cc2-adfd-19413119b4c5 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:56.777120 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 444/889] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:56.788552 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986ea88-f37a-463a-b9b2-83b4aed1f8f6 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:56.789180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986ea88-f37a-463a-b9b2-83b4aed1f8f6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:56.789764 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986ea88-f37a-463a-b9b2-83b4aed1f8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:56.810516 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-9986ea88-f37a-463a-b9b2-83b4aed1f8f6 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:56.812170 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9986ea88-f37a-463a-b9b2-83b4aed1f8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:56.820323 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22174aac-4707-4b9b-b38c-91ee6e1c7e14 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:56.865170 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebbbc2-7f78-4fcb-a9b7-47223a75cebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:56.865829 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebbbc2-7f78-4fcb-a9b7-47223a75cebc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:56.866085 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebbbc2-7f78-4fcb-a9b7-47223a75cebc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:56.866362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebbbc2-7f78-4fcb-a9b7-47223a75cebc None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:56.867425 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ebbbc2-7f78-4fcb-a9b7-47223a75cebc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:56.867735 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ebbbc2-7f78-4fcb-a9b7-47223a75cebc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:56.869459 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ebbbc2-7f78-4fcb-a9b7-47223a75cebc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:56.882762 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 446/890] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Sep 2 09:14:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:14:56.882762 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebbbc2-7f78-4fcb-a9b7-47223a75cebc None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:56.898175 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 445/891] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:56 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 09:14:56.898828 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986ea88-f37a-463a-b9b2-83b4aed1f8f6 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:56.945374 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:57.015485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:57.017622 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ebbbc2-7f78-4fcb-a9b7-47223a75cebc None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:57.019149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986ea88-f37a-463a-b9b2-83b4aed1f8f6 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:57.062628 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd64fe-fb00-4ce7-94d1-5f90ca151982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:57.063282 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd64fe-fb00-4ce7-94d1-5f90ca151982 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:57.063495 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967aa6dd-603e-46eb-9b70-e46f47124737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:57.063634 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd64fe-fb00-4ce7-94d1-5f90ca151982 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:57.063864 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd64fe-fb00-4ce7-94d1-5f90ca151982 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:57.064510 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fd64fe-fb00-4ce7-94d1-5f90ca151982 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:57.064795 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fd64fe-fb00-4ce7-94d1-5f90ca151982 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:57.065094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967aa6dd-603e-46eb-9b70-e46f47124737 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:57.065578 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967aa6dd-603e-46eb-9b70-e46f47124737 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:57.066038 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967aa6dd-603e-46eb-9b70-e46f47124737 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:57.066138 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fd64fe-fb00-4ce7-94d1-5f90ca151982 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:57.066917 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967aa6dd-603e-46eb-9b70-e46f47124737 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:57.069180 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967aa6dd-603e-46eb-9b70-e46f47124737 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:57.077608 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 446/892] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Sep 2 09:14:56 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:57.078728 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd64fe-fb00-4ce7-94d1-5f90ca151982 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:57.086084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-967aa6dd-603e-46eb-9b70-e46f47124737 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 09:14:57.086084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:57.086084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 09:14:57.086084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 09:14:57.086084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 09:14:57.086084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 09:14:57.086084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 09:14:57.086084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:14:57.086084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 09:14:57.086084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:57.086084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:14:57.086084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:57.086084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:57.086084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:57.086084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:57.086084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:57.086084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:57.087107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:57.087107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:57.087107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:57.087107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:57.087107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:57.087107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:57.087107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 09:14:57.087107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 09:14:57.087107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 09:14:57.087107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 09:14:57.087107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:57.087107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:57.087107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:57.087107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:57.087107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:57.087107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:57.087958 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:57.087958 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:57.087958 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:57.087958 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:57.087958 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:57.087958 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:57.087958 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:57.087958 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:57.087958 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:57.087958 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:57.087958 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 09:14:57.087958 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:14:57.087958 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 09:14:57.087958 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:57.087958 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 447/893] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Sep 2 09:14:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:57.093189 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967aa6dd-603e-46eb-9b70-e46f47124737 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:57.207226 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd64fe-fb00-4ce7-94d1-5f90ca151982 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:57.215335 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967aa6dd-603e-46eb-9b70-e46f47124737 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:57.224656 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:57.249922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0c1e3-bff6-4c03-86bd-f46c036c71e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:57.250297 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b61c1-ea8d-4174-ab41-915563f43161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:57.250742 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0c1e3-bff6-4c03-86bd-f46c036c71e6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:57.250933 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b61c1-ea8d-4174-ab41-915563f43161 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:57.251029 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0c1e3-bff6-4c03-86bd-f46c036c71e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:57.251196 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b61c1-ea8d-4174-ab41-915563f43161 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:57.251290 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c0c1e3-bff6-4c03-86bd-f46c036c71e6 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:57.251445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b61c1-ea8d-4174-ab41-915563f43161 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:57.251918 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0c1e3-bff6-4c03-86bd-f46c036c71e6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:57.252124 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b61c1-ea8d-4174-ab41-915563f43161 None admin] RBAC: Adding query filter params (interface=public, service_id=15c4fcaa94814cb2b4366dbc308e3e2b, region_id=RegionOne) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:57.252428 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b61c1-ea8d-4174-ab41-915563f43161 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:57.252524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0c1e3-bff6-4c03-86bd-f46c036c71e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:57.253768 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b61c1-ea8d-4174-ab41-915563f43161 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:57.254019 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c0c1e3-bff6-4c03-86bd-f46c036c71e6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:57.259924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 447/894] 10.222.0.57 () {56 vars in 1251 bytes} [Fri Sep 2 09:14:57 2022] GET /identity/v3/endpoints?service_id=15c4fcaa94814cb2b4366dbc308e3e2b&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:57.260093 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 448/895] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Sep 2 09:14:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:57.260174 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b61c1-ea8d-4174-ab41-915563f43161 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:57.265341 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0c1e3-bff6-4c03-86bd-f46c036c71e6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:57.289694 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:57.364308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b61c1-ea8d-4174-ab41-915563f43161 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:57.378868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c0c1e3-bff6-4c03-86bd-f46c036c71e6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:57.398045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88094b04-7e4d-4818-a1da-2e67eb63c178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:57.398666 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88094b04-7e4d-4818-a1da-2e67eb63c178 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:57.398985 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88094b04-7e4d-4818-a1da-2e67eb63c178 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:57.399254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88094b04-7e4d-4818-a1da-2e67eb63c178 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:57.409167 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88094b04-7e4d-4818-a1da-2e67eb63c178 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:14:57.414416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-88094b04-7e4d-4818-a1da-2e67eb63c178 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 09:14:57.414416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:57.414416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:57.414416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:57.414416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:57.414416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:57.414416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:57.414416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:57.414416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:57.414416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:57.414416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:57.414416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:57.414416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:14:57.414416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:14:57.414416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:14:57.414416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:14:57.415397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:14:57.415397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:14:57.415397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:14:57.415397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:14:57.415397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:57.415397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:57.415397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:57.415397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:57.415397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:57.415397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:57.415397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:57.415397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:57.415397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:57.415397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:57.415397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:57.415397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:57.416261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:57.416261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:57.416261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:57.416261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:57.416261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:14:57.416261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:14:57.416261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:14:57.416261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:14:57.416261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:14:57.416261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:14:57.416261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 09:14:57.416261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:57.416261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 448/896] 10.222.0.57 () {56 vars in 1120 bytes} [Fri Sep 2 09:14:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:57.416951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88094b04-7e4d-4818-a1da-2e67eb63c178 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:57.421503 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca28b9e8-00bb-486b-afc8-b41c479cb676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:57.422121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28b9e8-00bb-486b-afc8-b41c479cb676 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:57.422396 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28b9e8-00bb-486b-afc8-b41c479cb676 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:57.422660 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca28b9e8-00bb-486b-afc8-b41c479cb676 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:57.423310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca28b9e8-00bb-486b-afc8-b41c479cb676 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:57.424652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca28b9e8-00bb-486b-afc8-b41c479cb676 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:57.431629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 449/897] 10.222.0.57 () {56 vars in 1087 bytes} [Fri Sep 2 09:14:57 2022] GET /identity/v3/services => generated 1268 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:14:57.432542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca28b9e8-00bb-486b-afc8-b41c479cb676 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:57.487410 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:57.546063 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:57.565999 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88094b04-7e4d-4818-a1da-2e67eb63c178 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:57.576702 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca28b9e8-00bb-486b-afc8-b41c479cb676 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:57.615771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacc7b0-c6a1-4ce8-abfd-60927576752b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:57.616637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacc7b0-c6a1-4ce8-abfd-60927576752b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:57.616914 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacc7b0-c6a1-4ce8-abfd-60927576752b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:57.617180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacc7b0-c6a1-4ce8-abfd-60927576752b None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:57.617804 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bacc7b0-c6a1-4ce8-abfd-60927576752b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:57.618106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bacc7b0-c6a1-4ce8-abfd-60927576752b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:57.619473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bacc7b0-c6a1-4ce8-abfd-60927576752b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:57.621346 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb1cf0-5472-4019-8913-ee0c16ab6f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:57.622417 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb1cf0-5472-4019-8913-ee0c16ab6f34 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:57.622908 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb1cf0-5472-4019-8913-ee0c16ab6f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:57.623327 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb1cf0-5472-4019-8913-ee0c16ab6f34 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:57.624145 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb1cf0-5472-4019-8913-ee0c16ab6f34 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:57.624455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb1cf0-5472-4019-8913-ee0c16ab6f34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:57.625510 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 449/898] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Sep 2 09:14:57 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:57.625803 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb1cf0-5472-4019-8913-ee0c16ab6f34 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:57.639516 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 450/899] 10.222.0.57 () {56 vars in 1115 bytes} [Fri Sep 2 09:14:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:57.646763 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacc7b0-c6a1-4ce8-abfd-60927576752b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:57.758514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacc7b0-c6a1-4ce8-abfd-60927576752b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:57.796482 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d5aac-bd68-4a8a-8316-dc52e164c15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:57.797125 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d5aac-bd68-4a8a-8316-dc52e164c15f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:57.797405 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d5aac-bd68-4a8a-8316-dc52e164c15f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:57.797680 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d5aac-bd68-4a8a-8316-dc52e164c15f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:57.808783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d5aac-bd68-4a8a-8316-dc52e164c15f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:57.810193 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d5aac-bd68-4a8a-8316-dc52e164c15f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:57.825789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-346d5aac-bd68-4a8a-8316-dc52e164c15f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:57.825789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:57.825789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:57.825789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:57.825789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:57.825789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:57.825789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:57.825789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:57.825789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:57.825789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:57.825789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:57.825789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:57.825789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:57.825789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:57.825789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:57.825789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:57.826904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:57.826904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:57.826904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:57.826904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:57.826904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:57.826904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:57.826904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:57.826904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:57.826904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:57.826904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:57.826904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:57.826904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:57.826904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:57.826904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:57.826904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:57.826904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:57.826904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:57.827811 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:57.827811 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:57.827811 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:57.827811 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 450/900] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:57.833379 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb1cf0-5472-4019-8913-ee0c16ab6f34 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:57.941107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:57.945745 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb1cf0-5472-4019-8913-ee0c16ab6f34 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:57.981685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf09018-ef9a-42bc-abdc-5b7738a982fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:57.982355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf09018-ef9a-42bc-abdc-5b7738a982fd None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:57.982639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf09018-ef9a-42bc-abdc-5b7738a982fd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:57.982963 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf09018-ef9a-42bc-abdc-5b7738a982fd None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:57.983605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf09018-ef9a-42bc-abdc-5b7738a982fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:57.983907 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf09018-ef9a-42bc-abdc-5b7738a982fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:57.985316 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf09018-ef9a-42bc-abdc-5b7738a982fd None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:58.002155 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 451/901] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:58.003525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d5aac-bd68-4a8a-8316-dc52e164c15f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:58.006972 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:58.105794 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d5aac-bd68-4a8a-8316-dc52e164c15f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:58.139695 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609bf72-94ec-4fe8-acad-3d56d5c29a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:58.140332 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609bf72-94ec-4fe8-acad-3d56d5c29a64 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:58.140608 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609bf72-94ec-4fe8-acad-3d56d5c29a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:58.140877 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609bf72-94ec-4fe8-acad-3d56d5c29a64 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:58.149059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609bf72-94ec-4fe8-acad-3d56d5c29a64 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:58.150465 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609bf72-94ec-4fe8-acad-3d56d5c29a64 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:58.161740 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-3609bf72-94ec-4fe8-acad-3d56d5c29a64 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 09:14:58.161740 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:58.161740 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:58.161740 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:58.161740 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:58.161740 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:58.161740 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:58.161740 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:58.161740 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:58.161740 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:58.161740 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:58.161740 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:58.161740 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:14:58.161740 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:14:58.161740 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:14:58.161740 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:14:58.163996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:58.163996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:58.163996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:14:58.163996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:58.163996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:14:58.163996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:14:58.163996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:58.163996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:58.163996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:58.163996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:58.163996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:58.163996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:58.163996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:58.163996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:58.163996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:58.163996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:58.163996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:58.163996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:58.166207 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:58.166207 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:58.166207 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:14:58.166207 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:14:58.166207 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:14:58.166207 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:14:58.166207 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:14:58.166207 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:14:58.166207 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 09:14:58.166207 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:58.166207 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 451/902] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Sep 2 09:14:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:58.170737 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf09018-ef9a-42bc-abdc-5b7738a982fd None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:58.277425 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf09018-ef9a-42bc-abdc-5b7738a982fd None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:58.313425 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f54cfb-246c-4d36-8671-a9f4ea6776ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:58.314035 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f54cfb-246c-4d36-8671-a9f4ea6776ac None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:58.314310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f54cfb-246c-4d36-8671-a9f4ea6776ac None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:58.314580 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f54cfb-246c-4d36-8671-a9f4ea6776ac None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:58.315391 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f54cfb-246c-4d36-8671-a9f4ea6776ac None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:58.315696 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f54cfb-246c-4d36-8671-a9f4ea6776ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:58.317101 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f54cfb-246c-4d36-8671-a9f4ea6776ac None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:58.339759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 452/903] 10.222.0.57 () {56 vars in 1149 bytes} [Fri Sep 2 09:14:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:58.341134 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609bf72-94ec-4fe8-acad-3d56d5c29a64 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:58.404978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:58.445530 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609bf72-94ec-4fe8-acad-3d56d5c29a64 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:58.479946 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0efc557-f10f-4fa6-8932-2ad45f0be7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:58.480557 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efc557-f10f-4fa6-8932-2ad45f0be7c1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:58.480837 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efc557-f10f-4fa6-8932-2ad45f0be7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:58.481112 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efc557-f10f-4fa6-8932-2ad45f0be7c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:58.550461 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0efc557-f10f-4fa6-8932-2ad45f0be7c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:58.552483 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0efc557-f10f-4fa6-8932-2ad45f0be7c1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:58.613774 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:58.656574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 452/904] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:14:58 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 09:14:58.662604 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f54cfb-246c-4d36-8671-a9f4ea6776ac None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:58.681762 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:58.769252 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5a0ca-2f41-4471-8ea0-3b216ced2878 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:58.769602 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f54cfb-246c-4d36-8671-a9f4ea6776ac None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:58.769783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5a0ca-2f41-4471-8ea0-3b216ced2878 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:58.769783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5a0ca-2f41-4471-8ea0-3b216ced2878 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:58.771358 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 453/905] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:58.776147 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d235f-1762-4dca-858f-fca683bedf29 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:58.776147 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d235f-1762-4dca-858f-fca683bedf29 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:58.776486 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d235f-1762-4dca-858f-fca683bedf29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:58.795570 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b38d235f-1762-4dca-858f-fca683bedf29 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:58.796759 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b38d235f-1762-4dca-858f-fca683bedf29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:58.810155 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5288f-8bc3-4283-b408-a0f853ac30fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:58.811206 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5288f-8bc3-4283-b408-a0f853ac30fa None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:58.811828 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5288f-8bc3-4283-b408-a0f853ac30fa None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:58.812299 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d5288f-8bc3-4283-b408-a0f853ac30fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:58.823472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d5288f-8bc3-4283-b408-a0f853ac30fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:58.825273 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d5288f-8bc3-4283-b408-a0f853ac30fa None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:58.840633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-65d5288f-8bc3-4283-b408-a0f853ac30fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:58.840633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:58.840633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:58.840633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:58.840633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:58.840633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:58.840633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:58.840633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:58.840633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:58.840633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:58.840633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:58.840633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:58.840633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:14:58.840633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:14:58.840633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:14:58.840633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:14:58.842008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:58.842008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:58.842008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:58.842008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:58.842008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:58.842008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:58.842008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:58.842008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:58.842008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:58.842008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:58.842008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:58.842008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:58.842008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:58.842008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:58.842008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:58.842008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:58.842008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:14:58.843366 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:14:58.843366 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:14:58.843366 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:58.845753 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 453/906] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:14:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:58.853212 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5288f-8bc3-4283-b408-a0f853ac30fa None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:58.874840 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 454/907] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:58 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 09:14:58.931176 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:58.961912 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d5288f-8bc3-4283-b408-a0f853ac30fa None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:58.998753 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25620d01-45e8-4eb2-9032-9184785e4d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:58.998753 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25620d01-45e8-4eb2-9032-9184785e4d46 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:58.998753 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25620d01-45e8-4eb2-9032-9184785e4d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:58.998753 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25620d01-45e8-4eb2-9032-9184785e4d46 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:59.002622 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25620d01-45e8-4eb2-9032-9184785e4d46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:59.003073 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25620d01-45e8-4eb2-9032-9184785e4d46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:59.004655 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25620d01-45e8-4eb2-9032-9184785e4d46 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:59.027508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 454/908] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:59.030008 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d235f-1762-4dca-858f-fca683bedf29 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:59.187280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:59.196156 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d235f-1762-4dca-858f-fca683bedf29 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:59.201846 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25620d01-45e8-4eb2-9032-9184785e4d46 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:59.231918 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05024b17-d483-4e19-9c2c-a4cc861a0e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:59.232865 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05024b17-d483-4e19-9c2c-a4cc861a0e9a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:59.233149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05024b17-d483-4e19-9c2c-a4cc861a0e9a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:59.233424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05024b17-d483-4e19-9c2c-a4cc861a0e9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:59.242042 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05024b17-d483-4e19-9c2c-a4cc861a0e9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:59.243508 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05024b17-d483-4e19-9c2c-a4cc861a0e9a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:59.257696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-05024b17-d483-4e19-9c2c-a4cc861a0e9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:59.257696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:59.257696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:59.257696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:59.257696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:59.257696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:59.257696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:59.257696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:59.257696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:59.257696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:59.257696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:59.257696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:59.257696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:14:59.257696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:14:59.257696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:14:59.258605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:14:59.258605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:59.258605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:59.258605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:59.258605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:59.258605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:59.258605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:59.258605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:59.258605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:59.258605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:59.258605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:59.258605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:59.258605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:59.258605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:59.258605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:59.258605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:59.258605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:59.259862 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:14:59.259862 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:14:59.259862 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:14:59.259862 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:14:59.259862 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:14:59.259862 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:14:59.259862 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:14:59.259862 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:14:59.259862 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 455/909] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:14:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:14:59.265502 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05024b17-d483-4e19-9c2c-a4cc861a0e9a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:59.380989 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25620d01-45e8-4eb2-9032-9184785e4d46 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:59.434510 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:59.442107 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05024b17-d483-4e19-9c2c-a4cc861a0e9a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:59.454322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062d5394-5375-4ade-b900-c3735ddc6f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:59.455567 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d5394-5375-4ade-b900-c3735ddc6f98 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:59.456033 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d5394-5375-4ade-b900-c3735ddc6f98 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:59.456467 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d5394-5375-4ade-b900-c3735ddc6f98 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:59.457193 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062d5394-5375-4ade-b900-c3735ddc6f98 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:59.458816 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062d5394-5375-4ade-b900-c3735ddc6f98 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:59.480819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-062d5394-5375-4ade-b900-c3735ddc6f98 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 02 09:14:59.480819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:59.480819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 09:14:59.480819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 09:14:59.480819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 09:14:59.480819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 09:14:59.480819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 09:14:59.480819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:14:59.480819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 02 09:14:59.480819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:59.480819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:14:59.480819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:59.480819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:14:59.480819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:14:59.480819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:14:59.480819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:14:59.481905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:14:59.481905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:14:59.481905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:14:59.481905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:14:59.481905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:14:59.481905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:14:59.481905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:14:59.481905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 09:14:59.481905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 09:14:59.481905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 09:14:59.481905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 09:14:59.481905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:14:59.481905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:14:59.481905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:14:59.481905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:14:59.481905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:14:59.481905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:14:59.482791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:14:59.482791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:14:59.482791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:14:59.482791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:14:59.482791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:14:59.482791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:14:59.482791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:14:59.482791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:14:59.482791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:14:59.482791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:14:59.482791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 09:14:59.482791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:14:59.482791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 02 09:14:59.482791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:14:59.494088 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 455/910] 10.222.0.57 () {56 vars in 1132 bytes} [Fri Sep 2 09:14:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:14:59.495319 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062d5394-5375-4ade-b900-c3735ddc6f98 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:59.511957 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da16ab-abd4-418d-8ca8-06461334a1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:59.512544 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da16ab-abd4-418d-8ca8-06461334a1f6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:59.512819 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da16ab-abd4-418d-8ca8-06461334a1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:59.513088 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da16ab-abd4-418d-8ca8-06461334a1f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:59.513775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da16ab-abd4-418d-8ca8-06461334a1f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:59.514075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da16ab-abd4-418d-8ca8-06461334a1f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:59.515515 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da16ab-abd4-418d-8ca8-06461334a1f6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:59.530757 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 456/911] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:14:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:59.533744 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da16ab-abd4-418d-8ca8-06461334a1f6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:59.554655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:59.617727 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062d5394-5375-4ade-b900-c3735ddc6f98 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:59.642080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da16ab-abd4-418d-8ca8-06461334a1f6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:59.650490 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffcb810-aa7e-4ec8-a2a1-14dfb9d19c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:59.651139 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcb810-aa7e-4ec8-a2a1-14dfb9d19c1a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:59.651423 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcb810-aa7e-4ec8-a2a1-14dfb9d19c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:59.651697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcb810-aa7e-4ec8-a2a1-14dfb9d19c1a None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:59.652682 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffcb810-aa7e-4ec8-a2a1-14dfb9d19c1a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:59.652987 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffcb810-aa7e-4ec8-a2a1-14dfb9d19c1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:59.654655 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffcb810-aa7e-4ec8-a2a1-14dfb9d19c1a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:59.659552 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 456/912] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Sep 2 09:14:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:59.660683 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b38ba-f8b7-4cfe-9262-aed6ca26a3ee None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:59.661484 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b38ba-f8b7-4cfe-9262-aed6ca26a3ee None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:59.661758 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b38ba-f8b7-4cfe-9262-aed6ca26a3ee None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:59.662998 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 457/913] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:59.667859 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f8b4c-f2d8-4e5c-9ae2-beb54a54a0f6 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:59.668144 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f8b4c-f2d8-4e5c-9ae2-beb54a54a0f6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:59.668432 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7f8b4c-f2d8-4e5c-9ae2-beb54a54a0f6 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:59.669663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 458/914] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:14:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:14:59.670358 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f8b4c-f2d8-4e5c-9ae2-beb54a54a0f6 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:59.679352 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e30ca9-72d1-4202-a0d1-8b2784d609ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:59.680363 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e30ca9-72d1-4202-a0d1-8b2784d609ef None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:59.680645 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e30ca9-72d1-4202-a0d1-8b2784d609ef None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:59.680918 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e30ca9-72d1-4202-a0d1-8b2784d609ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:59.681660 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e30ca9-72d1-4202-a0d1-8b2784d609ef None admin] RBAC: Adding query filter params (role.id=dd2879946b3d4351acbaf8698b7966e8, scope.project.id=d18a5117a03b4f21ab3290a15680de66, user.id=5233689a53364258b86ad2fa012d8db1) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:59.681971 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e30ca9-72d1-4202-a0d1-8b2784d609ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:59.683413 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e30ca9-72d1-4202-a0d1-8b2784d609ef None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:59.688680 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 457/915] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:14:59 2022] GET /identity/v3/role_assignments?user.id=5233689a53364258b86ad2fa012d8db1&scope.project.id=d18a5117a03b4f21ab3290a15680de66&role.id=dd2879946b3d4351acbaf8698b7966e8 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:14:59.689845 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106eae7-0d12-4904-83f3-5f2825ee779c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:59.690130 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106eae7-0d12-4904-83f3-5f2825ee779c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:59.690400 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106eae7-0d12-4904-83f3-5f2825ee779c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:59.709711 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4106eae7-0d12-4904-83f3-5f2825ee779c None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:59.711461 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4106eae7-0d12-4904-83f3-5f2825ee779c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:59.749725 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:59.787694 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 458/916] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:59 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:14:59.789318 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a3c1c-013c-4ddd-a887-102f1c20b8a8 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:59.789824 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a3c1c-013c-4ddd-a887-102f1c20b8a8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:59.790291 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a3c1c-013c-4ddd-a887-102f1c20b8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:59.803235 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:14:59.810204 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-711a3c1c-013c-4ddd-a887-102f1c20b8a8 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:14:59.811905 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-711a3c1c-013c-4ddd-a887-102f1c20b8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:14:59.812043 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7f8b4c-f2d8-4e5c-9ae2-beb54a54a0f6 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:14:59.873519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423e987-0441-4cf0-97f4-413ac6216601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:14:59.875152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423e987-0441-4cf0-97f4-413ac6216601 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:14:59.876014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423e987-0441-4cf0-97f4-413ac6216601 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:14:59.876485 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423e987-0441-4cf0-97f4-413ac6216601 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:14:59.877298 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a423e987-0441-4cf0-97f4-413ac6216601 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:14:59.877826 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a423e987-0441-4cf0-97f4-413ac6216601 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:14:59.879498 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a423e987-0441-4cf0-97f4-413ac6216601 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:14:59.885692 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 459/917] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Sep 2 09:14:59 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:14:59.891379 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423e987-0441-4cf0-97f4-413ac6216601 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:14:59.905318 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 459/918] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:14:59 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:15:00.001209 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423e987-0441-4cf0-97f4-413ac6216601 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:00.022251 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:00.037416 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0854f80-bffc-4bcb-ae78-11a6c1c54dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:00.045623 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0854f80-bffc-4bcb-ae78-11a6c1c54dd3 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:00.045623 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0854f80-bffc-4bcb-ae78-11a6c1c54dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:00.045623 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0854f80-bffc-4bcb-ae78-11a6c1c54dd3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:00.045623 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0854f80-bffc-4bcb-ae78-11a6c1c54dd3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:00.045623 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0854f80-bffc-4bcb-ae78-11a6c1c54dd3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:00.066191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a3c1c-013c-4ddd-a887-102f1c20b8a8 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:00.143236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 460/919] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:14:59 2022] POST /identity/v3/endpoints => generated 347 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:15:00.176943 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0854f80-bffc-4bcb-ae78-11a6c1c54dd3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:00.185776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a3c1c-013c-4ddd-a887-102f1c20b8a8 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:00.235223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:00.241683 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e5d16-421e-4f0c-92a8-2073f50d112f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:00.242392 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e5d16-421e-4f0c-92a8-2073f50d112f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:00.242997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e5d16-421e-4f0c-92a8-2073f50d112f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:00.243279 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e5d16-421e-4f0c-92a8-2073f50d112f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:00.243877 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e5d16-421e-4f0c-92a8-2073f50d112f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:00.246274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e5d16-421e-4f0c-92a8-2073f50d112f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:00.260855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-b65e5d16-421e-4f0c-92a8-2073f50d112f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 02 09:15:00.260855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:00.260855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 09:15:00.260855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 09:15:00.260855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 09:15:00.260855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 09:15:00.260855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 09:15:00.260855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:15:00.260855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 02 09:15:00.260855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:00.260855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:15:00.260855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:00.260855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:00.260855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:00.260855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:00.260855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:00.260855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:00.261881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:00.261881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:00.261881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:00.261881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:00.261881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:00.261881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:00.261881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 09:15:00.261881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 09:15:00.261881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 09:15:00.261881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 09:15:00.261881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:00.261881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:00.261881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:00.261881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:00.261881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:00.261881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:00.262766 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:00.262766 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:00.262766 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:00.262766 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:00.262766 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:00.262766 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:00.262766 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:00.262766 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:00.262766 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:00.262766 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:00.262766 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 09:15:00.262766 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:15:00.262766 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 02 09:15:00.262766 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:00.262766 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 460/920] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Sep 2 09:15:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:15:00.268019 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e5d16-421e-4f0c-92a8-2073f50d112f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:00.320455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0854f80-bffc-4bcb-ae78-11a6c1c54dd3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:00.363792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25196eb2-52d6-48b7-84d8-ea61347b3aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:00.364557 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25196eb2-52d6-48b7-84d8-ea61347b3aaa None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:00.364906 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25196eb2-52d6-48b7-84d8-ea61347b3aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:00.365589 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25196eb2-52d6-48b7-84d8-ea61347b3aaa None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:00.366341 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25196eb2-52d6-48b7-84d8-ea61347b3aaa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:00.368406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25196eb2-52d6-48b7-84d8-ea61347b3aaa None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:00.391018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e5d16-421e-4f0c-92a8-2073f50d112f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:00.426742 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084dc543-fd23-409a-b351-3e9c26774f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:00.427725 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084dc543-fd23-409a-b351-3e9c26774f50 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:00.428185 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084dc543-fd23-409a-b351-3e9c26774f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:00.428638 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084dc543-fd23-409a-b351-3e9c26774f50 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:00.429438 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084dc543-fd23-409a-b351-3e9c26774f50 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:00.429918 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084dc543-fd23-409a-b351-3e9c26774f50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:00.431536 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084dc543-fd23-409a-b351-3e9c26774f50 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:00.437074 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 461/921] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Sep 2 09:15:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:00.444421 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084dc543-fd23-409a-b351-3e9c26774f50 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:00.446369 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 461/922] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Sep 2 09:15:00 2022] POST /identity/v3/services => generated 241 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:15:00.521451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:00.572967 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084dc543-fd23-409a-b351-3e9c26774f50 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:00.616997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851a919-2e38-4a15-a268-b5949a866bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:00.617619 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851a919-2e38-4a15-a268-b5949a866bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:00.617904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851a919-2e38-4a15-a268-b5949a866bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:00.618179 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851a919-2e38-4a15-a268-b5949a866bd7 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:00.619409 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851a919-2e38-4a15-a268-b5949a866bd7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:00.619409 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851a919-2e38-4a15-a268-b5949a866bd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:00.620569 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851a919-2e38-4a15-a268-b5949a866bd7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:00.626207 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 462/923] 10.222.0.57 () {56 vars in 1114 bytes} [Fri Sep 2 09:15:00 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:00.632396 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25196eb2-52d6-48b7-84d8-ea61347b3aaa None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:00.674406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:00.742636 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25196eb2-52d6-48b7-84d8-ea61347b3aaa None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:00.779701 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97129030-af02-43cb-b86c-f89d99d1f4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:00.780663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97129030-af02-43cb-b86c-f89d99d1f4e7 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:00.781160 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97129030-af02-43cb-b86c-f89d99d1f4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:00.781626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97129030-af02-43cb-b86c-f89d99d1f4e7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:00.782526 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97129030-af02-43cb-b86c-f89d99d1f4e7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:00.784111 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97129030-af02-43cb-b86c-f89d99d1f4e7 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:00.916063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 462/924] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:15:00 2022] POST /identity/v3/endpoints => generated 346 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:15:01.648519 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd04009-cf06-45a1-aa89-0c2f0e99d4fe None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:01.649090 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd04009-cf06-45a1-aa89-0c2f0e99d4fe None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:01.649362 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd04009-cf06-45a1-aa89-0c2f0e99d4fe None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:01.651100 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 463/925] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:01.656286 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275fd50-d745-4971-b8c4-367851a73062 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:01.656707 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275fd50-d745-4971-b8c4-367851a73062 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:01.657086 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c275fd50-d745-4971-b8c4-367851a73062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:01.675613 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c275fd50-d745-4971-b8c4-367851a73062 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:01.676831 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c275fd50-d745-4971-b8c4-367851a73062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:01.752874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 463/926] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:15:02.022417 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd04009-cf06-45a1-aa89-0c2f0e99d4fe None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:02.105290 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d28a37-cb2d-4267-a48a-99b70c7b6d90 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:02.105801 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d28a37-cb2d-4267-a48a-99b70c7b6d90 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:02.105877 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d28a37-cb2d-4267-a48a-99b70c7b6d90 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:02.107453 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 464/927] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:02.112418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea3d01-9e55-42d3-b104-37c3f88090b6 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:02.112706 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea3d01-9e55-42d3-b104-37c3f88090b6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:02.112977 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea3d01-9e55-42d3-b104-37c3f88090b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:02.132537 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-49ea3d01-9e55-42d3-b104-37c3f88090b6 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:02.133929 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-49ea3d01-9e55-42d3-b104-37c3f88090b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:02.137869 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd04009-cf06-45a1-aa89-0c2f0e99d4fe None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:02.148288 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:02.158903 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:02.173191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1d2cb-78ce-43b2-9da1-e85e3426c89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:02.174120 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1d2cb-78ce-43b2-9da1-e85e3426c89a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:02.174595 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1d2cb-78ce-43b2-9da1-e85e3426c89a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:02.175090 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1d2cb-78ce-43b2-9da1-e85e3426c89a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:02.186694 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb1d2cb-78ce-43b2-9da1-e85e3426c89a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:15:02.191765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-4fb1d2cb-78ce-43b2-9da1-e85e3426c89a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 09:15:02.191765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:02.191765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:02.191765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:02.191765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:02.191765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:02.191765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:02.191765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:02.191765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:02.191765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:02.191765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:02.191765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:02.191765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:15:02.191765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:15:02.191765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:15:02.191765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:15:02.195887 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:15:02.195887 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:15:02.195887 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:15:02.195887 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:15:02.195887 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:02.195887 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:02.195887 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:02.195887 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:02.195887 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:02.195887 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:02.195887 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:02.195887 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:02.195887 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:02.195887 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:02.195887 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:02.195887 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:02.196997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:02.196997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:02.196997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:02.196997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:02.196997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:15:02.196997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:15:02.196997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:15:02.196997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:15:02.196997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:15:02.196997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:15:02.196997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 09:15:02.196997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:02.196997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 464/928] 10.222.0.57 () {56 vars in 1120 bytes} [Fri Sep 2 09:15:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:02.200871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1d2cb-78ce-43b2-9da1-e85e3426c89a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:02.213355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 465/929] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:15:02.305336 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1d2cb-78ce-43b2-9da1-e85e3426c89a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:02.340227 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09433087-b513-4a86-b7b6-684a8f123387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:02.341161 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09433087-b513-4a86-b7b6-684a8f123387 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:02.341677 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09433087-b513-4a86-b7b6-684a8f123387 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:02.342161 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09433087-b513-4a86-b7b6-684a8f123387 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:02.343050 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09433087-b513-4a86-b7b6-684a8f123387 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:02.343534 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09433087-b513-4a86-b7b6-684a8f123387 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:02.345062 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09433087-b513-4a86-b7b6-684a8f123387 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:02.357990 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 465/930] 10.222.0.57 () {56 vars in 1115 bytes} [Fri Sep 2 09:15:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:02.363609 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea3d01-9e55-42d3-b104-37c3f88090b6 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:02.474894 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea3d01-9e55-42d3-b104-37c3f88090b6 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:02.509894 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcb833-b4e4-40ac-99a7-0328645db6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:02.510530 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcb833-b4e4-40ac-99a7-0328645db6b1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:02.511107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcb833-b4e4-40ac-99a7-0328645db6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:02.511553 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fcb833-b4e4-40ac-99a7-0328645db6b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:02.513145 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6709c5-fd2a-4b38-b87b-271fa1225032 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:02.513438 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6709c5-fd2a-4b38-b87b-271fa1225032 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:02.513720 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6709c5-fd2a-4b38-b87b-271fa1225032 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:02.515354 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 466/931] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:02.521130 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed501fc6-a433-4d03-a0b6-187be9874d01 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:02.521657 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed501fc6-a433-4d03-a0b6-187be9874d01 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:02.522137 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed501fc6-a433-4d03-a0b6-187be9874d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:02.523547 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fcb833-b4e4-40ac-99a7-0328645db6b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:02.524978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fcb833-b4e4-40ac-99a7-0328645db6b1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:02.536322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:02.543821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-b4fcb833-b4e4-40ac-99a7-0328645db6b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:15:02.543821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:02.543821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:02.543821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:02.543821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:02.543821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:02.543821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:02.543821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:02.543821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:02.543821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:02.543821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:02.543821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:02.543821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:15:02.543821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:15:02.543821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:15:02.543821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:15:02.550259 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:02.550259 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:02.550259 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:02.550259 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:02.550259 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:02.550259 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:02.550259 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:02.550259 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:02.550259 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:02.550259 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:02.550259 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:02.550259 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:02.550259 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:02.550259 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:02.550259 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:02.550259 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:02.550259 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:15:02.551275 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:15:02.551275 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:15:02.551275 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:02.551275 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 466/932] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:15:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:02.551518 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ed501fc6-a433-4d03-a0b6-187be9874d01 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:02.551602 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcb833-b4e4-40ac-99a7-0328645db6b1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:02.551673 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ed501fc6-a433-4d03-a0b6-187be9874d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:02.587707 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:02.619401 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 467/933] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:15:02.620533 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed501fc6-a433-4d03-a0b6-187be9874d01 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:02.668333 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fcb833-b4e4-40ac-99a7-0328645db6b1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:02.715240 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77a3c1-b736-4bb4-9c01-9575223c3ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:02.716217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77a3c1-b736-4bb4-9c01-9575223c3ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:02.716700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77a3c1-b736-4bb4-9c01-9575223c3ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:02.717169 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e77a3c1-b736-4bb4-9c01-9575223c3ff6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:02.717968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77a3c1-b736-4bb4-9c01-9575223c3ff6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:02.719609 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e77a3c1-b736-4bb4-9c01-9575223c3ff6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:02.728875 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-6e77a3c1-b736-4bb4-9c01-9575223c3ff6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 02 09:15:02.728875 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:02.728875 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 09:15:02.728875 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 09:15:02.728875 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 09:15:02.728875 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 09:15:02.728875 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 09:15:02.728875 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:15:02.728875 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 02 09:15:02.728875 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:02.728875 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:15:02.728875 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:02.728875 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:02.728875 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:02.728875 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:02.728875 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:02.728875 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:02.729879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:02.729879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:02.729879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:02.729879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:02.729879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:02.729879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:02.729879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 09:15:02.729879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 09:15:02.729879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 09:15:02.729879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 09:15:02.729879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:02.729879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:02.729879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:02.729879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:02.729879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:02.729879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:02.730745 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:02.730745 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:02.730745 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:02.730745 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:02.730745 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:02.730745 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:02.730745 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:02.730745 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:02.730745 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:02.730745 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:02.730745 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 09:15:02.730745 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:15:02.730745 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 02 09:15:02.730745 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:02.733487 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 467/934] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:15:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:15:02.740064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77a3c1-b736-4bb4-9c01-9575223c3ff6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:02.747687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed501fc6-a433-4d03-a0b6-187be9874d01 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:02.790770 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d90ab-6ec7-4147-b066-2161ccb6bddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:02.791729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d90ab-6ec7-4147-b066-2161ccb6bddc None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:02.792210 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d90ab-6ec7-4147-b066-2161ccb6bddc None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:02.793111 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d90ab-6ec7-4147-b066-2161ccb6bddc None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:02.793960 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d90ab-6ec7-4147-b066-2161ccb6bddc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:02.794463 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d90ab-6ec7-4147-b066-2161ccb6bddc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:02.796685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d90ab-6ec7-4147-b066-2161ccb6bddc None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:02.811908 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 468/935] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:02.818348 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d90ab-6ec7-4147-b066-2161ccb6bddc None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:02.824693 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:02.865648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:02.873819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e77a3c1-b736-4bb4-9c01-9575223c3ff6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:02.909457 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e80d8-3b6d-46c2-92ac-44cc020e5a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:02.910071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e80d8-3b6d-46c2-92ac-44cc020e5a78 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:02.910352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e80d8-3b6d-46c2-92ac-44cc020e5a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:02.910644 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623e80d8-3b6d-46c2-92ac-44cc020e5a78 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:02.911304 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e80d8-3b6d-46c2-92ac-44cc020e5a78 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:02.911608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e80d8-3b6d-46c2-92ac-44cc020e5a78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:02.912956 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623e80d8-3b6d-46c2-92ac-44cc020e5a78 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:02.917484 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 468/936] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Sep 2 09:15:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:02.918265 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e80d8-3b6d-46c2-92ac-44cc020e5a78 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:02.949392 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d90ab-6ec7-4147-b066-2161ccb6bddc None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:02.981975 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ad623-c371-4c41-8a20-e7b9ea37ccd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:02.982592 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ad623-c371-4c41-8a20-e7b9ea37ccd4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:02.982903 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ad623-c371-4c41-8a20-e7b9ea37ccd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:02.983181 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ad623-c371-4c41-8a20-e7b9ea37ccd4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:02.991444 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ad623-c371-4c41-8a20-e7b9ea37ccd4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:02.992861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ad623-c371-4c41-8a20-e7b9ea37ccd4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:03.003776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-e66ad623-c371-4c41-8a20-e7b9ea37ccd4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 09:15:03.003776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:03.003776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:03.003776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:03.003776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:03.003776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:03.003776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:03.003776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:03.003776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:03.003776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:03.003776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:03.003776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:03.003776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:15:03.003776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:15:03.003776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:15:03.003776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:15:03.005449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:03.005449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:03.005449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:15:03.005449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:15:03.005449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:15:03.005449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:15:03.005449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:03.005449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:03.005449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:03.005449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:03.005449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:03.005449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:03.005449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:03.005449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:03.005449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:03.005449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:03.005449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:03.005449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:03.006407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:03.006407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:03.006407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:15:03.006407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:15:03.006407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:15:03.006407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:15:03.006407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:15:03.006407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:15:03.006407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 09:15:03.006407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:03.006407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 469/937] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Sep 2 09:15:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:15:03.007041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234de676-a0e1-4c02-ac90-c987d1083540 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:03.007534 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234de676-a0e1-4c02-ac90-c987d1083540 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:03.007534 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234de676-a0e1-4c02-ac90-c987d1083540 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:03.008817 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 470/938] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:03.009413 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234de676-a0e1-4c02-ac90-c987d1083540 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:03.023390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623e80d8-3b6d-46c2-92ac-44cc020e5a78 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:03.048809 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:03.057009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638d55c-7176-46eb-b0e0-f507ff332433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:03.059913 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638d55c-7176-46eb-b0e0-f507ff332433 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:03.059913 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638d55c-7176-46eb-b0e0-f507ff332433 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:03.059913 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5638d55c-7176-46eb-b0e0-f507ff332433 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:03.059913 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5638d55c-7176-46eb-b0e0-f507ff332433 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:03.060208 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5638d55c-7176-46eb-b0e0-f507ff332433 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:03.069121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-5638d55c-7176-46eb-b0e0-f507ff332433 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 09:15:03.069121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:03.069121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 09:15:03.069121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 09:15:03.069121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 09:15:03.069121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 09:15:03.069121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 09:15:03.069121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:15:03.069121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 09:15:03.069121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:03.069121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:15:03.069121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:03.069121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:03.069121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:03.069121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:03.069121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:03.069121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:03.070125 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:03.070125 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:03.070125 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:03.070125 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:03.070125 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:03.070125 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:03.070125 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 09:15:03.070125 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 09:15:03.070125 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 09:15:03.070125 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 09:15:03.070125 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:03.070125 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:03.070125 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:03.070125 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:03.070125 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:03.070125 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:03.071475 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:03.071475 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:03.071475 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:03.071475 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:03.071475 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:03.071475 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:03.071475 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:03.071475 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:03.071475 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:03.071475 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:03.071475 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 09:15:03.071475 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:15:03.071475 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 09:15:03.071475 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:03.071475 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 469/939] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Sep 2 09:15:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:03.072298 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638d55c-7176-46eb-b0e0-f507ff332433 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:03.104366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:03.114466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234de676-a0e1-4c02-ac90-c987d1083540 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:03.148068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598cce17-a626-4eaf-92a5-4099c57a138b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:03.148672 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cce17-a626-4eaf-92a5-4099c57a138b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:03.148960 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cce17-a626-4eaf-92a5-4099c57a138b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:03.149234 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598cce17-a626-4eaf-92a5-4099c57a138b None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:03.149858 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598cce17-a626-4eaf-92a5-4099c57a138b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:03.150163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598cce17-a626-4eaf-92a5-4099c57a138b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:03.151559 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598cce17-a626-4eaf-92a5-4099c57a138b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:03.155964 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 471/940] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Sep 2 09:15:03 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:03.157167 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fad09-f111-441b-ace2-696920d98660 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:03.157452 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fad09-f111-441b-ace2-696920d98660 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:03.157721 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fad09-f111-441b-ace2-696920d98660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:03.177610 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b94fad09-f111-441b-ace2-696920d98660 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:03.178877 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b94fad09-f111-441b-ace2-696920d98660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:03.183355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5638d55c-7176-46eb-b0e0-f507ff332433 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:03.220395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75098841-9ee3-4f37-800c-4c50e1424420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:03.221024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75098841-9ee3-4f37-800c-4c50e1424420 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:03.221300 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75098841-9ee3-4f37-800c-4c50e1424420 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:03.221568 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75098841-9ee3-4f37-800c-4c50e1424420 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:03.222315 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75098841-9ee3-4f37-800c-4c50e1424420 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:03.222624 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75098841-9ee3-4f37-800c-4c50e1424420 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:03.224634 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75098841-9ee3-4f37-800c-4c50e1424420 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:03.244129 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 470/941] 10.222.0.57 () {56 vars in 1149 bytes} [Fri Sep 2 09:15:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:03.245001 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75098841-9ee3-4f37-800c-4c50e1424420 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:03.259319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 472/942] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:15:03.260568 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94fad09-f111-441b-ace2-696920d98660 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:03.317139 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:03.378496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:03.387984 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75098841-9ee3-4f37-800c-4c50e1424420 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:03.395802 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94fad09-f111-441b-ace2-696920d98660 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:03.432814 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d564e9-3755-42e0-ac7a-1f78b7ab79de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:03.433426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d564e9-3755-42e0-ac7a-1f78b7ab79de None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:03.433704 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d564e9-3755-42e0-ac7a-1f78b7ab79de None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:03.433971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d564e9-3755-42e0-ac7a-1f78b7ab79de None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:03.434907 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d564e9-3755-42e0-ac7a-1f78b7ab79de None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:03.435209 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d564e9-3755-42e0-ac7a-1f78b7ab79de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:03.436830 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d564e9-3755-42e0-ac7a-1f78b7ab79de None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:03.442998 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 471/943] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Sep 2 09:15:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:03.443705 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc444565-8959-4c3a-9d7c-ef57eaef077d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:03.444627 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc444565-8959-4c3a-9d7c-ef57eaef077d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:03.445128 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc444565-8959-4c3a-9d7c-ef57eaef077d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:03.445608 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc444565-8959-4c3a-9d7c-ef57eaef077d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:03.449557 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d564e9-3755-42e0-ac7a-1f78b7ab79de None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:03.457820 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc444565-8959-4c3a-9d7c-ef57eaef077d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:03.459717 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc444565-8959-4c3a-9d7c-ef57eaef077d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:03.476216 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-bc444565-8959-4c3a-9d7c-ef57eaef077d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:15:03.476216 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:03.476216 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:03.476216 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:03.476216 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:03.476216 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:03.476216 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:03.476216 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:03.476216 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:03.476216 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:03.476216 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:03.476216 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:03.476216 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:15:03.476216 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:15:03.476216 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:15:03.476216 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:15:03.477165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:03.477165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:03.477165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:03.477165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:03.477165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:03.477165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:03.477165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:03.477165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:03.477165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:03.477165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:03.477165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:03.477165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:03.477165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:03.477165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:03.477165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:03.477165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:03.477165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:15:03.478055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:15:03.478055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:15:03.478055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:03.480298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 473/944] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:15:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:03.494731 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc444565-8959-4c3a-9d7c-ef57eaef077d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:03.576518 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d564e9-3755-42e0-ac7a-1f78b7ab79de None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:03.589540 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc444565-8959-4c3a-9d7c-ef57eaef077d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:03.591411 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:03.612235 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f6dcc-06ee-4419-b39f-0de6e27aeb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:03.612847 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f6dcc-06ee-4419-b39f-0de6e27aeb87 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:03.613127 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f6dcc-06ee-4419-b39f-0de6e27aeb87 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:03.613396 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f6dcc-06ee-4419-b39f-0de6e27aeb87 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:03.614554 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6f6dcc-06ee-4419-b39f-0de6e27aeb87 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:03.614883 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6f6dcc-06ee-4419-b39f-0de6e27aeb87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:03.616495 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6f6dcc-06ee-4419-b39f-0de6e27aeb87 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:03.622505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 472/945] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Sep 2 09:15:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:03.623139 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f6dcc-06ee-4419-b39f-0de6e27aeb87 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:03.635302 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0806ded-f1d4-4b7e-8062-70b60405d68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:03.635853 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0806ded-f1d4-4b7e-8062-70b60405d68d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:03.636139 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0806ded-f1d4-4b7e-8062-70b60405d68d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:03.636416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0806ded-f1d4-4b7e-8062-70b60405d68d None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:03.637355 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0806ded-f1d4-4b7e-8062-70b60405d68d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:03.637667 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0806ded-f1d4-4b7e-8062-70b60405d68d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:03.639059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0806ded-f1d4-4b7e-8062-70b60405d68d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:03.646696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:03.651752 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 474/946] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:15:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:15:03.652623 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0806ded-f1d4-4b7e-8062-70b60405d68d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:03.740989 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f6dcc-06ee-4419-b39f-0de6e27aeb87 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:03.771191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0806ded-f1d4-4b7e-8062-70b60405d68d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:03.777835 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360feea-11e0-42f3-96d9-335d399d35ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:03.778458 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360feea-11e0-42f3-96d9-335d399d35ab None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:03.778772 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360feea-11e0-42f3-96d9-335d399d35ab None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:03.779049 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360feea-11e0-42f3-96d9-335d399d35ab None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:03.779702 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360feea-11e0-42f3-96d9-335d399d35ab None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:03.781126 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4360feea-11e0-42f3-96d9-335d399d35ab None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:03.783810 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: WARNING keystone.server.flask.common [None req-4360feea-11e0-42f3-96d9-335d399d35ab None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 02 09:15:03.808692 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0dc7fc-3a3c-464a-ac64-0e5f5a6aa4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:03.809305 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dc7fc-3a3c-464a-ac64-0e5f5a6aa4aa None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:03.809587 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dc7fc-3a3c-464a-ac64-0e5f5a6aa4aa None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:03.809864 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0dc7fc-3a3c-464a-ac64-0e5f5a6aa4aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:03.816648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0dc7fc-3a3c-464a-ac64-0e5f5a6aa4aa None admin] RBAC: Adding query filter params (interface=public, service_id=c9cb416f214a46ad9571659959580616, region_id=RegionOne) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:03.816648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0dc7fc-3a3c-464a-ac64-0e5f5a6aa4aa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:03.816648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0dc7fc-3a3c-464a-ac64-0e5f5a6aa4aa None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:03.817733 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 475/947] 10.222.0.57 () {56 vars in 1251 bytes} [Fri Sep 2 09:15:03 2022] GET /identity/v3/endpoints?service_id=c9cb416f214a46ad9571659959580616&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:03.818503 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0dc7fc-3a3c-464a-ac64-0e5f5a6aa4aa None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:03.827952 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.sql.core [None req-4360feea-11e0-42f3-96d9-335d399d35ab None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 02 09:15:03.827952 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 02 09:15:03.827952 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [parameters: {'id': 'aadda2bd9d954cec9d200dbafe958f0e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 02 09:15:03.827952 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81611) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 02 09:15:03.836583 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4360feea-11e0-42f3-96d9-335d399d35ab None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id aadda2bd9d954cec9d200dbafe958f0e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id aadda2bd9d954cec9d200dbafe958f0e. Sep 02 09:15:03.836583 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:03.836583 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 02 09:15:03.836583 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 02 09:15:03.836583 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 02 09:15:03.836583 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 02 09:15:03.836583 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 02 09:15:03.836583 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result = self._query(query) Sep 02 09:15:03.836583 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 02 09:15:03.836583 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application conn.query(q) Sep 02 09:15:03.836583 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 02 09:15:03.836583 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 02 09:15:03.836583 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 02 09:15:03.837561 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result.read() Sep 02 09:15:03.837561 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 02 09:15:03.837561 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 02 09:15:03.837561 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 02 09:15:03.837561 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 02 09:15:03.837561 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 02 09:15:03.837561 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 02 09:15:03.837561 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 02 09:15:03.837561 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 02 09:15:03.837561 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 02 09:15:03.837561 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:03.837561 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 02 09:15:03.837561 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:03.837561 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:03.837561 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 02 09:15:03.837561 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 02 09:15:03.837561 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 02 09:15:03.837561 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 02 09:15:03.838668 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 02 09:15:03.838668 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application next(self.gen) Sep 02 09:15:03.838668 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 02 09:15:03.838668 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application yield resource Sep 02 09:15:03.838668 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 02 09:15:03.838668 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application next(self.gen) Sep 02 09:15:03.838668 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 02 09:15:03.838668 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.session.rollback() Sep 02 09:15:03.838668 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 02 09:15:03.838668 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.force_reraise() Sep 02 09:15:03.838668 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 02 09:15:03.838668 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise self.value Sep 02 09:15:03.838668 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 02 09:15:03.838668 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 02 09:15:03.838668 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 02 09:15:03.838668 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application session.commit() Sep 02 09:15:03.838668 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 02 09:15:03.839709 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 02 09:15:03.839709 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 02 09:15:03.839709 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._prepare_impl() Sep 02 09:15:03.839709 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 02 09:15:03.839709 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.session.flush() Sep 02 09:15:03.839709 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 02 09:15:03.839709 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._flush(objects) Sep 02 09:15:03.839709 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 02 09:15:03.839709 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 02 09:15:03.839709 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 02 09:15:03.839709 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application compat.raise_( Sep 02 09:15:03.839709 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 02 09:15:03.839709 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception Sep 02 09:15:03.839709 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 02 09:15:03.839709 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application flush_context.execute() Sep 02 09:15:03.839709 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 02 09:15:03.839709 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rec.execute(self) Sep 02 09:15:03.840598 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 02 09:15:03.840598 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 02 09:15:03.840598 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 02 09:15:03.840598 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 02 09:15:03.840598 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 02 09:15:03.840598 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 02 09:15:03.840598 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 02 09:15:03.840598 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 02 09:15:03.840598 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 02 09:15:03.840598 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 02 09:15:03.840598 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 02 09:15:03.840598 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 02 09:15:03.840598 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 02 09:15:03.840598 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 02 09:15:03.840598 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 02 09:15:03.841382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 02 09:15:03.841382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 02 09:15:03.841382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception Sep 02 09:15:03.841382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 02 09:15:03.841382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 02 09:15:03.841382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 02 09:15:03.841382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 02 09:15:03.841382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 02 09:15:03.841382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result = self._query(query) Sep 02 09:15:03.841382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 02 09:15:03.841382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application conn.query(q) Sep 02 09:15:03.841382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 02 09:15:03.841382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 02 09:15:03.841382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 02 09:15:03.841382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result.read() Sep 02 09:15:03.841382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 02 09:15:03.841382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 02 09:15:03.842638 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 02 09:15:03.842638 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 02 09:15:03.842638 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 02 09:15:03.842638 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 02 09:15:03.842638 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 02 09:15:03.842638 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 02 09:15:03.842638 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 02 09:15:03.842638 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 02 09:15:03.842638 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [parameters: {'id': 'aadda2bd9d954cec9d200dbafe958f0e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 02 09:15:03.842638 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 02 09:15:03.842638 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:03.842638 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:15:03.842638 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:03.842638 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:03.842638 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 02 09:15:03.843663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 02 09:15:03.843663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 02 09:15:03.843663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 02 09:15:03.843663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 02 09:15:03.843663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:03.843663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:15:03.843663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:03.843663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:03.843663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:03.843663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:03.843663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:03.843663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:03.843663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:03.843663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:03.843663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:03.843663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:03.843663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:03.844804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:03.844804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 02 09:15:03.844804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 02 09:15:03.844804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:03.844804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:03.844804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 02 09:15:03.844804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 02 09:15:03.844804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id aadda2bd9d954cec9d200dbafe958f0e. Sep 02 09:15:03.844804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:03.844804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 473/948] 10.222.0.57 () {60 vars in 1140 bytes} [Fri Sep 2 09:15:03 2022] POST /identity/v3/projects => generated 262 bytes in 215 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:03.844804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360feea-11e0-42f3-96d9-335d399d35ab None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:03.880989 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:03.898217 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:03.938131 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0dc7fc-3a3c-464a-ac64-0e5f5a6aa4aa None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:03.951548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360feea-11e0-42f3-96d9-335d399d35ab None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:03.975428 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fe7db-a2e1-41e7-b7c1-c1b68aecb032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:03.976086 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fe7db-a2e1-41e7-b7c1-c1b68aecb032 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:03.976367 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fe7db-a2e1-41e7-b7c1-c1b68aecb032 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:03.976640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696fe7db-a2e1-41e7-b7c1-c1b68aecb032 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:03.983330 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696fe7db-a2e1-41e7-b7c1-c1b68aecb032 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:03.984760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696fe7db-a2e1-41e7-b7c1-c1b68aecb032 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:03.986116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb1f06-a691-4052-80b6-d9c69db6a5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:03.986756 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb1f06-a691-4052-80b6-d9c69db6a5d1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:03.987024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb1f06-a691-4052-80b6-d9c69db6a5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:03.987297 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cb1f06-a691-4052-80b6-d9c69db6a5d1 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:03.987898 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb1f06-a691-4052-80b6-d9c69db6a5d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:03.989481 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cb1f06-a691-4052-80b6-d9c69db6a5d1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:03.996727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-696fe7db-a2e1-41e7-b7c1-c1b68aecb032 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:15:03.996727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:03.996727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:03.996727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:03.996727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:03.996727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:03.996727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:03.996727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:03.996727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:03.996727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:03.996727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:03.996727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:03.996727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:15:03.996727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:15:03.996727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:15:03.997621 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:15:03.997621 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:03.997621 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:03.997621 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:03.997621 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:03.997621 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:03.997621 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:03.997621 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:03.997621 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:03.997621 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:03.997621 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:03.997621 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:03.997621 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:03.997621 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:03.997621 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:03.997621 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:03.997621 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:03.998495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:15:03.998495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:15:03.998495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:15:03.998495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:15:03.998495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:15:03.998495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:15:03.998495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:15:03.998495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:04.002840 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 476/949] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:15:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:04.003537 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fe7db-a2e1-41e7-b7c1-c1b68aecb032 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:04.011128 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 474/950] 10.222.0.57 () {56 vars in 1087 bytes} [Fri Sep 2 09:15:03 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:15:04.022859 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb1f06-a691-4052-80b6-d9c69db6a5d1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:04.125041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696fe7db-a2e1-41e7-b7c1-c1b68aecb032 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:04.146472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:04.151332 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:04.154413 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cb1f06-a691-4052-80b6-d9c69db6a5d1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:04.165087 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d8250-ff03-4d18-a82a-38e8a4a74c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:04.165743 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d8250-ff03-4d18-a82a-38e8a4a74c9c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:04.166027 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d8250-ff03-4d18-a82a-38e8a4a74c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:04.166352 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d8250-ff03-4d18-a82a-38e8a4a74c9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:04.174544 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39d8250-ff03-4d18-a82a-38e8a4a74c9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:04.176022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39d8250-ff03-4d18-a82a-38e8a4a74c9c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:04.188706 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-b39d8250-ff03-4d18-a82a-38e8a4a74c9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:15:04.188706 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:04.188706 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:04.188706 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:04.188706 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:04.188706 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:04.188706 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:04.188706 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:04.188706 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:04.188706 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:04.188706 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:04.188706 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:04.188706 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:15:04.188706 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:15:04.188706 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:15:04.189671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:15:04.189671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:04.189671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:04.189671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:04.189671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:04.189671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:04.189671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:04.189671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:04.189671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:04.189671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:04.189671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:04.189671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:04.189671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:04.189671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:04.189671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:04.189671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:04.189671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:04.190626 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:15:04.190626 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:15:04.190626 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:15:04.190626 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:15:04.190626 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:15:04.190626 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:15:04.190626 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:15:04.190626 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:04.190626 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 477/951] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:15:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:04.197823 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d8250-ff03-4d18-a82a-38e8a4a74c9c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:04.203569 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed20d4-cae8-4101-a9c9-35704939eebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:04.204263 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed20d4-cae8-4101-a9c9-35704939eebf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:04.204613 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed20d4-cae8-4101-a9c9-35704939eebf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:04.204955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed20d4-cae8-4101-a9c9-35704939eebf None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:04.205728 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ed20d4-cae8-4101-a9c9-35704939eebf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:04.206102 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ed20d4-cae8-4101-a9c9-35704939eebf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:04.208223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ed20d4-cae8-4101-a9c9-35704939eebf None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:04.222093 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 475/952] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:15:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:04.227917 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed20d4-cae8-4101-a9c9-35704939eebf None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:04.319338 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d8250-ff03-4d18-a82a-38e8a4a74c9c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:04.342158 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed20d4-cae8-4101-a9c9-35704939eebf None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:04.355040 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232ca99-ee50-4794-abe9-b818b4e0fc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:04.355928 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232ca99-ee50-4794-abe9-b818b4e0fc48 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:04.356302 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232ca99-ee50-4794-abe9-b818b4e0fc48 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:04.356670 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232ca99-ee50-4794-abe9-b818b4e0fc48 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:04.357517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5232ca99-ee50-4794-abe9-b818b4e0fc48 None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:04.357984 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5232ca99-ee50-4794-abe9-b818b4e0fc48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:04.359722 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5232ca99-ee50-4794-abe9-b818b4e0fc48 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:04.378768 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 478/953] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Sep 2 09:15:04 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:15:04.389355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656f70c5-a35b-49aa-98e1-d1e74aaae43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:04.394911 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f70c5-a35b-49aa-98e1-d1e74aaae43c None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:04.394911 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f70c5-a35b-49aa-98e1-d1e74aaae43c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:04.394911 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656f70c5-a35b-49aa-98e1-d1e74aaae43c None admin] PATH_INFO: `/v3/projects/d18a5117a03b4f21ab3290a15680de66/users/5233689a53364258b86ad2fa012d8db1/roles/dd2879946b3d4351acbaf8698b7966e8` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:04.397522 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f70c5-a35b-49aa-98e1-d1e74aaae43c None admin] RBAC: Authorizing `identity:create_grant(project_id=d18a5117a03b4f21ab3290a15680de66, user_id=5233689a53364258b86ad2fa012d8db1, role_id=dd2879946b3d4351acbaf8698b7966e8)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:04.399242 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656f70c5-a35b-49aa-98e1-d1e74aaae43c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:04.498435 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 476/954] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Sep 2 09:15:04 2022] PUT /identity/v3/projects/d18a5117a03b4f21ab3290a15680de66/users/5233689a53364258b86ad2fa012d8db1/roles/dd2879946b3d4351acbaf8698b7966e8 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:05.110177 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9030fce-8bdd-400f-8cc6-ddc2a4adcc38 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:05.110690 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9030fce-8bdd-400f-8cc6-ddc2a4adcc38 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:05.110690 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9030fce-8bdd-400f-8cc6-ddc2a4adcc38 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:05.112313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 479/955] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:05.117504 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348422f-517b-44a4-bc73-e6ece07f6ebe None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:05.118018 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348422f-517b-44a4-bc73-e6ece07f6ebe None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:05.118520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348422f-517b-44a4-bc73-e6ece07f6ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:05.136999 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9348422f-517b-44a4-bc73-e6ece07f6ebe None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:05.138848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9348422f-517b-44a4-bc73-e6ece07f6ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:05.152973 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:05.210182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 477/956] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:15:05.478565 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9030fce-8bdd-400f-8cc6-ddc2a4adcc38 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:05.524615 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:05.600723 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9030fce-8bdd-400f-8cc6-ddc2a4adcc38 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:05.637567 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9fd47-edda-4d09-b956-2f1fad59c27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:05.638179 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9fd47-edda-4d09-b956-2f1fad59c27f None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:05.638518 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9fd47-edda-4d09-b956-2f1fad59c27f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:05.638829 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9fd47-edda-4d09-b956-2f1fad59c27f None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:05.639503 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9fd47-edda-4d09-b956-2f1fad59c27f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:05.640920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9fd47-edda-4d09-b956-2f1fad59c27f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:05.714809 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 480/957] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Sep 2 09:15:05 2022] POST /identity/v3/services => generated 242 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:15:06.050688 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493fb96-451b-4727-8c95-19da9c070b3e None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:06.051999 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493fb96-451b-4727-8c95-19da9c070b3e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:06.052488 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493fb96-451b-4727-8c95-19da9c070b3e None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:06.054339 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 478/958] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:06.060053 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e4557-0a06-4717-8ab7-043b40813011 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:06.060952 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e4557-0a06-4717-8ab7-043b40813011 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:06.061442 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e4557-0a06-4717-8ab7-043b40813011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:06.080787 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-059e4557-0a06-4717-8ab7-043b40813011 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:06.082353 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-059e4557-0a06-4717-8ab7-043b40813011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:06.161361 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 481/959] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:06 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:15:06.406832 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa611307-c831-4fd7-8922-a07bd7af9324 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:06.407443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa611307-c831-4fd7-8922-a07bd7af9324 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:06.407443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa611307-c831-4fd7-8922-a07bd7af9324 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:06.408948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 479/960] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:06.414369 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6226ff-352c-44c6-ab29-52162935f27d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:06.414603 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6226ff-352c-44c6-ab29-52162935f27d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:06.414929 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6226ff-352c-44c6-ab29-52162935f27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:06.433222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-cf6226ff-352c-44c6-ab29-52162935f27d None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:06.434420 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cf6226ff-352c-44c6-ab29-52162935f27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:06.436126 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa611307-c831-4fd7-8922-a07bd7af9324 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:06.460069 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:06.511009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 482/961] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:06 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:15:06.511009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b824a-3281-4877-9dd7-6c52f761da6c None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:06.511009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b824a-3281-4877-9dd7-6c52f761da6c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:06.511009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b824a-3281-4877-9dd7-6c52f761da6c None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:06.511009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 483/962] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:06.515097 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a568da5-8214-4927-8e71-3184c5d55029 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:06.515463 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a568da5-8214-4927-8e71-3184c5d55029 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:06.515884 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a568da5-8214-4927-8e71-3184c5d55029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:06.534620 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5a568da5-8214-4927-8e71-3184c5d55029 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:06.536621 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5a568da5-8214-4927-8e71-3184c5d55029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:06.549259 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa611307-c831-4fd7-8922-a07bd7af9324 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:06.589455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf4314f-2adb-4668-83ac-c62c4fdb8611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:06.590340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4314f-2adb-4668-83ac-c62c4fdb8611 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:06.590803 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4314f-2adb-4668-83ac-c62c4fdb8611 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:06.591217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4314f-2adb-4668-83ac-c62c4fdb8611 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:06.591962 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf4314f-2adb-4668-83ac-c62c4fdb8611 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:06.593517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf4314f-2adb-4668-83ac-c62c4fdb8611 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:06.603350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-4cf4314f-2adb-4668-83ac-c62c4fdb8611 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 09:15:06.603350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:06.603350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 09:15:06.603350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 09:15:06.603350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 09:15:06.603350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 09:15:06.603350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 09:15:06.603350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:15:06.603350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 09:15:06.603350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:06.603350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:15:06.603350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:06.603350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:06.603350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:06.603350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:06.603350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:06.603350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:06.610796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:06.610796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:06.610796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:06.610796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:06.610796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:06.610796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:06.610796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 09:15:06.610796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 09:15:06.610796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 09:15:06.610796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 09:15:06.610796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:06.610796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:06.610796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:06.610796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:06.610796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:06.610796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:06.611900 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:06.611900 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:06.611900 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:06.611900 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:06.611900 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:06.611900 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:06.611900 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:06.611900 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:06.611900 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:06.611900 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:06.611900 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 09:15:06.611900 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:15:06.611900 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 09:15:06.611900 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:06.613375 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 480/963] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Sep 2 09:15:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:15:06.614855 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf4314f-2adb-4668-83ac-c62c4fdb8611 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:06.631012 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:06.638902 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 484/964] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:06 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 09:15:06.725297 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf4314f-2adb-4668-83ac-c62c4fdb8611 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:06.759908 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5750841-87e0-49fc-bd3b-473d05650662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:06.760782 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5750841-87e0-49fc-bd3b-473d05650662 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:06.761219 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5750841-87e0-49fc-bd3b-473d05650662 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:06.761639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5750841-87e0-49fc-bd3b-473d05650662 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:06.762411 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5750841-87e0-49fc-bd3b-473d05650662 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:06.763073 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5750841-87e0-49fc-bd3b-473d05650662 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:06.764579 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5750841-87e0-49fc-bd3b-473d05650662 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:06.774799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 481/965] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Sep 2 09:15:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:06.776788 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a568da5-8214-4927-8e71-3184c5d55029 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:06.787960 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5750841-87e0-49fc-bd3b-473d05650662 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:06.876714 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:06.900879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5750841-87e0-49fc-bd3b-473d05650662 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:06.901314 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:06.914379 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a568da5-8214-4927-8e71-3184c5d55029 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:06.938137 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77e7e1-2f75-4702-8de0-01f694fe0a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:06.938966 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77e7e1-2f75-4702-8de0-01f694fe0a5f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:06.939253 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77e7e1-2f75-4702-8de0-01f694fe0a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:06.939854 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d77e7e1-2f75-4702-8de0-01f694fe0a5f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:06.950822 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1ddfa-10ad-4c0c-a685-d1bfd1558fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:06.951119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d77e7e1-2f75-4702-8de0-01f694fe0a5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:15:06.951425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1ddfa-10ad-4c0c-a685-d1bfd1558fbc None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:06.951705 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1ddfa-10ad-4c0c-a685-d1bfd1558fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:06.951972 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1ddfa-10ad-4c0c-a685-d1bfd1558fbc None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:06.952593 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e1ddfa-10ad-4c0c-a685-d1bfd1558fbc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:06.952894 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e1ddfa-10ad-4c0c-a685-d1bfd1558fbc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:06.954238 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e1ddfa-10ad-4c0c-a685-d1bfd1558fbc None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:06.956330 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-8d77e7e1-2f75-4702-8de0-01f694fe0a5f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 09:15:06.956330 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:06.956330 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:06.956330 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:06.956330 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:06.956330 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:06.956330 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:06.956330 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:06.956330 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:06.956330 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:06.956330 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:06.956330 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:06.956330 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:15:06.956330 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:15:06.956330 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:15:06.956330 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:15:06.957472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:15:06.957472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:15:06.957472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:15:06.957472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:15:06.957472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:06.957472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:06.957472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:06.957472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:06.957472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:06.957472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:06.957472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:06.957472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:06.957472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:06.957472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:06.957472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:06.957472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:06.958912 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:06.958912 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:06.958912 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:06.958912 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:06.958912 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:15:06.958912 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:15:06.958912 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:15:06.958912 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:15:06.958912 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:15:06.958912 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:15:06.958912 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 09:15:06.958912 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:06.958912 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 482/966] 10.222.0.57 () {56 vars in 1120 bytes} [Fri Sep 2 09:15:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:15:06.958912 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77e7e1-2f75-4702-8de0-01f694fe0a5f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:06.970674 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 485/967] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Sep 2 09:15:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:06.970674 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1ddfa-10ad-4c0c-a685-d1bfd1558fbc None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:07.062505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d77e7e1-2f75-4702-8de0-01f694fe0a5f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:07.074745 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1ddfa-10ad-4c0c-a685-d1bfd1558fbc None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:07.101825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d3ce1-695d-4dd4-80ae-526f40dd14ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:07.103202 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d3ce1-695d-4dd4-80ae-526f40dd14ae None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:07.103798 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d3ce1-695d-4dd4-80ae-526f40dd14ae None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:07.104362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d3ce1-695d-4dd4-80ae-526f40dd14ae None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:07.105326 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252d3ce1-695d-4dd4-80ae-526f40dd14ae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:07.107515 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252d3ce1-695d-4dd4-80ae-526f40dd14ae None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:07.110448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c72c99-95bf-4bdc-8d1e-cc5687d6f701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:07.111084 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c72c99-95bf-4bdc-8d1e-cc5687d6f701 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:07.111369 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c72c99-95bf-4bdc-8d1e-cc5687d6f701 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:07.111642 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c72c99-95bf-4bdc-8d1e-cc5687d6f701 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:07.112305 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c72c99-95bf-4bdc-8d1e-cc5687d6f701 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:07.112609 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c72c99-95bf-4bdc-8d1e-cc5687d6f701 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:07.114611 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c72c99-95bf-4bdc-8d1e-cc5687d6f701 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:07.118638 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:07.123666 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 486/968] 10.222.0.57 () {56 vars in 1115 bytes} [Fri Sep 2 09:15:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:07.124332 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c72c99-95bf-4bdc-8d1e-cc5687d6f701 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:07.140151 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:07.194107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.sql.core [None req-252d3ce1-695d-4dd4-80ae-526f40dd14ae None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 02 09:15:07.194107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 02 09:15:07.194107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [parameters: {'user_id': 'c46c43dd237e4a8a917888572e78775e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 02 09:15:07.194107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81611) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 02 09:15:07.203216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-252d3ce1-695d-4dd4-80ae-526f40dd14ae None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 02 09:15:07.203216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:07.203216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 02 09:15:07.203216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 02 09:15:07.203216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 02 09:15:07.203216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 02 09:15:07.203216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 02 09:15:07.203216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result = self._query(query) Sep 02 09:15:07.203216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 02 09:15:07.203216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application conn.query(q) Sep 02 09:15:07.203216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 02 09:15:07.203216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 02 09:15:07.203216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 02 09:15:07.203216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result.read() Sep 02 09:15:07.203216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 02 09:15:07.204171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 02 09:15:07.204171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 02 09:15:07.204171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 02 09:15:07.204171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 02 09:15:07.204171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 02 09:15:07.204171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 02 09:15:07.204171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 02 09:15:07.204171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 02 09:15:07.204171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:07.204171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 02 09:15:07.204171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:07.204171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:07.204171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 02 09:15:07.204171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 02 09:15:07.204171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 02 09:15:07.204171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 02 09:15:07.204171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 02 09:15:07.204171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application next(self.gen) Sep 02 09:15:07.205171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 02 09:15:07.205171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application yield resource Sep 02 09:15:07.205171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 02 09:15:07.205171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application next(self.gen) Sep 02 09:15:07.205171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 02 09:15:07.205171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.session.rollback() Sep 02 09:15:07.205171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 02 09:15:07.205171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.force_reraise() Sep 02 09:15:07.205171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 02 09:15:07.205171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise self.value Sep 02 09:15:07.205171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 02 09:15:07.205171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 02 09:15:07.205171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 02 09:15:07.205171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application session.commit() Sep 02 09:15:07.205171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 02 09:15:07.205171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 02 09:15:07.205171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 02 09:15:07.206118 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._prepare_impl() Sep 02 09:15:07.206118 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 02 09:15:07.206118 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.session.flush() Sep 02 09:15:07.206118 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 02 09:15:07.206118 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._flush(objects) Sep 02 09:15:07.206118 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 02 09:15:07.206118 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 02 09:15:07.206118 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 02 09:15:07.206118 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application compat.raise_( Sep 02 09:15:07.206118 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 02 09:15:07.206118 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception Sep 02 09:15:07.206118 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 02 09:15:07.206118 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application flush_context.execute() Sep 02 09:15:07.206118 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 02 09:15:07.206118 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rec.execute(self) Sep 02 09:15:07.206118 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 02 09:15:07.206118 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 02 09:15:07.208257 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 02 09:15:07.208257 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 02 09:15:07.208257 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 02 09:15:07.208257 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 02 09:15:07.208257 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 02 09:15:07.208257 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 02 09:15:07.208257 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 02 09:15:07.208257 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 02 09:15:07.208257 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 02 09:15:07.208257 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 02 09:15:07.208257 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 02 09:15:07.208257 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 02 09:15:07.208257 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 02 09:15:07.208257 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 02 09:15:07.208257 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 02 09:15:07.208257 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception Sep 02 09:15:07.209362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 02 09:15:07.209362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 02 09:15:07.209362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 02 09:15:07.209362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 02 09:15:07.209362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 02 09:15:07.209362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result = self._query(query) Sep 02 09:15:07.209362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 02 09:15:07.209362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application conn.query(q) Sep 02 09:15:07.209362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 02 09:15:07.209362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 02 09:15:07.209362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 02 09:15:07.209362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application result.read() Sep 02 09:15:07.209362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 02 09:15:07.209362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 02 09:15:07.209362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 02 09:15:07.209362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 02 09:15:07.210548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 02 09:15:07.210548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 02 09:15:07.210548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 02 09:15:07.210548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 02 09:15:07.210548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 02 09:15:07.210548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 02 09:15:07.210548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c46c43dd237e4a8a917888572e78775e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 02 09:15:07.210548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 02 09:15:07.210548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:07.210548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:15:07.210548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:07.210548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:07.210548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:07.210548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:07.210548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:07.211605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:07.211605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:07.211605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:07.211605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:07.211605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:07.211605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:07.211605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:07.211605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 02 09:15:07.211605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 02 09:15:07.211605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:07.211605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:07.211605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:15:07.211605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:15:07.211605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:15:07.211605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:15:07.211605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 02 09:15:07.211605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 02 09:15:07.212855 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 02 09:15:07.212855 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 02 09:15:07.212855 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 02 09:15:07.212855 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 02 09:15:07.212855 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 02 09:15:07.212855 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:07.212855 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 483/969] 10.222.0.57 () {60 vars in 1131 bytes} [Fri Sep 2 09:15:06 2022] POST /identity/v3/users => generated 160 bytes in 246 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:07.212855 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d3ce1-695d-4dd4-80ae-526f40dd14ae None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:07.234596 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c72c99-95bf-4bdc-8d1e-cc5687d6f701 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:07.268807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fec600-dd4a-45ae-ba37-61c93cc300b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:07.269422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fec600-dd4a-45ae-ba37-61c93cc300b8 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:07.270185 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fec600-dd4a-45ae-ba37-61c93cc300b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:07.270454 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fec600-dd4a-45ae-ba37-61c93cc300b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:07.271158 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fec600-dd4a-45ae-ba37-61c93cc300b8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:07.272817 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fec600-dd4a-45ae-ba37-61c93cc300b8 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:07.312239 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d3ce1-695d-4dd4-80ae-526f40dd14ae None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:07.342270 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 487/970] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:15:07 2022] POST /identity/v3/endpoints => generated 349 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:15:07.342270 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fec600-dd4a-45ae-ba37-61c93cc300b8 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:07.368419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a3fba-8843-476e-bffe-1ff1c6b17998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:07.371013 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a3fba-8843-476e-bffe-1ff1c6b17998 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:07.371013 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a3fba-8843-476e-bffe-1ff1c6b17998 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:07.371013 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a3fba-8843-476e-bffe-1ff1c6b17998 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:07.382514 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389a3fba-8843-476e-bffe-1ff1c6b17998 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:07.384356 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389a3fba-8843-476e-bffe-1ff1c6b17998 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:07.403214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-389a3fba-8843-476e-bffe-1ff1c6b17998 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:15:07.403214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:07.403214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:07.403214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:07.403214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:07.403214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:07.403214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:07.403214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:07.403214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:07.403214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:07.403214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:07.403214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:07.403214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:15:07.403214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:15:07.403214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:15:07.403214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:15:07.404308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:07.404308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:07.404308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:07.404308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:07.404308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:07.404308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:07.404308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:07.404308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:07.404308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:07.404308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:07.404308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:07.404308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:07.404308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:07.404308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:07.404308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:07.404308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:07.404308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:15:07.405219 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:15:07.405219 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:15:07.405219 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:07.409535 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 484/971] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:15:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:07.411164 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a3fba-8843-476e-bffe-1ff1c6b17998 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:07.435629 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:07.463137 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:07.481897 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fec600-dd4a-45ae-ba37-61c93cc300b8 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:07.529967 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd0a45-87e6-4685-8d4d-250d7fca58e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:07.530459 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd0a45-87e6-4685-8d4d-250d7fca58e3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:07.530696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd0a45-87e6-4685-8d4d-250d7fca58e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:07.532226 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd0a45-87e6-4685-8d4d-250d7fca58e3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:07.538613 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a3fba-8843-476e-bffe-1ff1c6b17998 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:07.542180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd0a45-87e6-4685-8d4d-250d7fca58e3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:07.543650 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd0a45-87e6-4685-8d4d-250d7fca58e3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:07.555422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-e3dd0a45-87e6-4685-8d4d-250d7fca58e3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 02 09:15:07.555422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:07.555422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:07.555422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:07.555422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:07.555422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:07.555422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:07.555422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:07.555422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:07.555422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:07.555422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:07.555422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:07.555422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:15:07.555422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:15:07.555422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:15:07.555422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:15:07.556644 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:07.556644 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:07.556644 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:15:07.556644 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:15:07.556644 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:15:07.556644 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:15:07.556644 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:07.556644 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:07.556644 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:07.556644 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:07.556644 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:07.556644 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:07.556644 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:07.556644 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:07.556644 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:07.556644 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:07.556644 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:07.556644 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:07.557859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:07.557859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:07.557859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:15:07.557859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:15:07.557859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:15:07.557859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:15:07.557859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:15:07.557859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:15:07.557859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 02 09:15:07.557859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:07.557859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 488/972] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:15:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:07.562685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd0a45-87e6-4685-8d4d-250d7fca58e3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:07.584062 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9040da58-eda6-4852-890b-7bd9371e51da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:07.585045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040da58-eda6-4852-890b-7bd9371e51da None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:07.585537 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040da58-eda6-4852-890b-7bd9371e51da None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:07.586111 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040da58-eda6-4852-890b-7bd9371e51da None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:07.587141 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9040da58-eda6-4852-890b-7bd9371e51da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:07.587648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9040da58-eda6-4852-890b-7bd9371e51da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:07.589191 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9040da58-eda6-4852-890b-7bd9371e51da None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:07.604426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 485/973] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:07.611705 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9040da58-eda6-4852-890b-7bd9371e51da None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:07.679287 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd0a45-87e6-4685-8d4d-250d7fca58e3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:07.710854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:07.716484 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e731e08-76e6-41d9-82ee-41976965ccf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:07.717098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e731e08-76e6-41d9-82ee-41976965ccf0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:07.717384 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e731e08-76e6-41d9-82ee-41976965ccf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:07.717660 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e731e08-76e6-41d9-82ee-41976965ccf0 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:07.718364 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e731e08-76e6-41d9-82ee-41976965ccf0 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:07.718671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e731e08-76e6-41d9-82ee-41976965ccf0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:07.720319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e731e08-76e6-41d9-82ee-41976965ccf0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:07.724776 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9040da58-eda6-4852-890b-7bd9371e51da None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:07.730880 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:07.763965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 489/974] 10.222.0.57 () {56 vars in 1101 bytes} [Fri Sep 2 09:15:07 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:15:07.763965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded58e9-3ed4-4440-84fe-a8664c7e2a25 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:07.763965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded58e9-3ed4-4440-84fe-a8664c7e2a25 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:07.763965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded58e9-3ed4-4440-84fe-a8664c7e2a25 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:07.763965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 490/975] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:07.766049 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5222c0a-c2f5-4938-84fd-d1840123a4fc None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:07.766268 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5222c0a-c2f5-4938-84fd-d1840123a4fc None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:07.766547 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5222c0a-c2f5-4938-84fd-d1840123a4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:07.767792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d0970-9a30-4d58-b41b-fc6934f71f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:07.768448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d0970-9a30-4d58-b41b-fc6934f71f0e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:07.768727 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d0970-9a30-4d58-b41b-fc6934f71f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:07.769066 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d0970-9a30-4d58-b41b-fc6934f71f0e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:07.778249 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d0970-9a30-4d58-b41b-fc6934f71f0e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:07.779760 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d0970-9a30-4d58-b41b-fc6934f71f0e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:07.786789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c5222c0a-c2f5-4938-84fd-d1840123a4fc None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:07.788030 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c5222c0a-c2f5-4938-84fd-d1840123a4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:07.792869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-1a0d0970-9a30-4d58-b41b-fc6934f71f0e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 09:15:07.792869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:07.792869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:07.792869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:07.792869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:07.792869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:07.792869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:07.792869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:07.792869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:07.792869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:07.792869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:07.792869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:07.792869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:15:07.792869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:15:07.792869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:15:07.792869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:15:07.793868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:07.793868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:07.793868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:15:07.793868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:15:07.793868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:15:07.793868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:15:07.793868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:07.793868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:07.793868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:07.793868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:07.793868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:07.793868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:07.793868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:07.793868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:07.793868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:07.793868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:07.793868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:07.793868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:07.794936 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:07.794936 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:07.794936 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:15:07.794936 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:15:07.794936 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:15:07.794936 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:15:07.794936 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:15:07.794936 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:15:07.794936 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 02 09:15:07.794936 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:07.794936 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 486/976] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Sep 2 09:15:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:07.800147 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d0970-9a30-4d58-b41b-fc6934f71f0e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:07.884061 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 491/977] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:07 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 09:15:07.933168 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d0970-9a30-4d58-b41b-fc6934f71f0e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:07.970894 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2fd597-d852-41b2-946d-f4c48002a6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:07.971509 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2fd597-d852-41b2-946d-f4c48002a6fa None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:07.971792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2fd597-d852-41b2-946d-f4c48002a6fa None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:07.972065 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2fd597-d852-41b2-946d-f4c48002a6fa None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:07.972854 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2fd597-d852-41b2-946d-f4c48002a6fa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:07.973397 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2fd597-d852-41b2-946d-f4c48002a6fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:07.974919 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2fd597-d852-41b2-946d-f4c48002a6fa None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:07.994886 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 487/978] 10.222.0.57 () {56 vars in 1149 bytes} [Fri Sep 2 09:15:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:15:08.000580 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5222c0a-c2f5-4938-84fd-d1840123a4fc None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:08.106221 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5222c0a-c2f5-4938-84fd-d1840123a4fc None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:08.123897 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:08.141368 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b933df5-2576-4e39-a913-e67b69a3a4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:08.141914 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b933df5-2576-4e39-a913-e67b69a3a4b9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:08.142194 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b933df5-2576-4e39-a913-e67b69a3a4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:08.142465 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b933df5-2576-4e39-a913-e67b69a3a4b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:08.166125 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2fd597-d852-41b2-946d-f4c48002a6fa None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:08.191450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:08.218463 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b933df5-2576-4e39-a913-e67b69a3a4b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:08.219946 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b933df5-2576-4e39-a913-e67b69a3a4b9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:08.270624 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2fd597-d852-41b2-946d-f4c48002a6fa None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:08.304575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc04e3-a2c0-4b3e-ba87-843a68128164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:08.305208 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc04e3-a2c0-4b3e-ba87-843a68128164 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:08.305489 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc04e3-a2c0-4b3e-ba87-843a68128164 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:08.305762 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc04e3-a2c0-4b3e-ba87-843a68128164 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:08.306343 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bc04e3-a2c0-4b3e-ba87-843a68128164 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:08.307743 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bc04e3-a2c0-4b3e-ba87-843a68128164 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:08.316393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-46bc04e3-a2c0-4b3e-ba87-843a68128164 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 02 09:15:08.316393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:08.316393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 09:15:08.316393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 09:15:08.316393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 09:15:08.316393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 09:15:08.316393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 09:15:08.316393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:15:08.316393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 02 09:15:08.316393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:08.316393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:15:08.316393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:08.316393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:08.316393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:08.316393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:08.316393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:08.316393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:08.318026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:08.318026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:08.318026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:08.318026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:08.318026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:08.318026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:08.318026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 09:15:08.318026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 09:15:08.318026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 09:15:08.318026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 09:15:08.318026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:08.318026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:08.318026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:08.318026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:08.318026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:08.318026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:08.318916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:08.318916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:08.318916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:08.318916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:08.318916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:08.318916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:08.318916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:08.318916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:08.318916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:08.318916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:08.318916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 09:15:08.318916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:15:08.318916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 02 09:15:08.318916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:08.318916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 488/979] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:15:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:15:08.323939 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc04e3-a2c0-4b3e-ba87-843a68128164 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:08.331048 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 492/980] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:15:07 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 02 09:15:08.331769 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b933df5-2576-4e39-a913-e67b69a3a4b9 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:08.350940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:08.421462 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:08.433925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc04e3-a2c0-4b3e-ba87-843a68128164 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:08.440634 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b933df5-2576-4e39-a913-e67b69a3a4b9 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:08.469953 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865306b-36af-4fc3-b944-10efc9ce77b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:08.470648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865306b-36af-4fc3-b944-10efc9ce77b5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:08.470956 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865306b-36af-4fc3-b944-10efc9ce77b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:08.471226 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0865306b-36af-4fc3-b944-10efc9ce77b5 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:08.471846 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0865306b-36af-4fc3-b944-10efc9ce77b5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:08.472147 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0865306b-36af-4fc3-b944-10efc9ce77b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:08.474016 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0865306b-36af-4fc3-b944-10efc9ce77b5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:08.477156 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff0fd94-24a2-4bf3-af97-2cde68376400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:08.477789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0fd94-24a2-4bf3-af97-2cde68376400 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:08.478067 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0fd94-24a2-4bf3-af97-2cde68376400 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:08.478341 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0fd94-24a2-4bf3-af97-2cde68376400 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:08.479568 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 489/981] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Sep 2 09:15:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:15:08.485309 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865306b-36af-4fc3-b944-10efc9ce77b5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:08.493643 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff0fd94-24a2-4bf3-af97-2cde68376400 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:08.495116 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff0fd94-24a2-4bf3-af97-2cde68376400 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:08.510489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-dff0fd94-24a2-4bf3-af97-2cde68376400 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:15:08.510489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:08.510489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:08.510489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:08.510489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:08.510489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:08.510489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:08.510489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:08.510489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:08.510489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:08.510489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:08.510489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:08.510489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:15:08.510489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:15:08.510489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:15:08.510489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:15:08.511679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:08.511679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:08.511679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:08.511679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:08.511679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:08.511679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:08.511679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:08.511679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:08.511679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:08.511679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:08.511679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:08.511679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:08.511679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:08.511679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:08.511679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:08.511679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:08.511679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:15:08.512803 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:15:08.512803 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:15:08.512803 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:08.519032 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 493/982] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:15:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:08.520381 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff0fd94-24a2-4bf3-af97-2cde68376400 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:08.592563 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0865306b-36af-4fc3-b944-10efc9ce77b5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:08.595820 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:08.628320 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a5160-4e4a-4fcb-b28c-a774a92c85c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:08.628951 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a5160-4e4a-4fcb-b28c-a774a92c85c5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:08.629240 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a5160-4e4a-4fcb-b28c-a774a92c85c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:08.629356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff0fd94-24a2-4bf3-af97-2cde68376400 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:08.629518 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a5160-4e4a-4fcb-b28c-a774a92c85c5 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:08.630142 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a5160-4e4a-4fcb-b28c-a774a92c85c5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:08.630446 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a5160-4e4a-4fcb-b28c-a774a92c85c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:08.631903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25a5160-4e4a-4fcb-b28c-a774a92c85c5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:08.636671 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 490/983] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Sep 2 09:15:08 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:15:08.642382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a5160-4e4a-4fcb-b28c-a774a92c85c5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:08.663997 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:08.665603 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d69661-188e-408a-b943-8a87ba4369fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:08.666007 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d69661-188e-408a-b943-8a87ba4369fe None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:08.666290 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d69661-188e-408a-b943-8a87ba4369fe None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:08.666564 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d69661-188e-408a-b943-8a87ba4369fe None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:08.667308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d69661-188e-408a-b943-8a87ba4369fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:08.667615 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d69661-188e-408a-b943-8a87ba4369fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:08.668965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d69661-188e-408a-b943-8a87ba4369fe None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:08.685868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 494/984] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:15:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:08.686825 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d69661-188e-408a-b943-8a87ba4369fe None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:08.754925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a5160-4e4a-4fcb-b28c-a774a92c85c5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:08.791358 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ef28a-ad83-4c5f-a87d-e5703cde5813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:08.791579 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d69661-188e-408a-b943-8a87ba4369fe None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:08.791976 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ef28a-ad83-4c5f-a87d-e5703cde5813 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:08.792258 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ef28a-ad83-4c5f-a87d-e5703cde5813 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:08.792531 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ef28a-ad83-4c5f-a87d-e5703cde5813 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:08.793232 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ef28a-ad83-4c5f-a87d-e5703cde5813 None admin] RBAC: Adding query filter params (interface=public, service_id=2bea28b9432f4cfaaf9d3312ed015958, region_id=RegionOne) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:08.793539 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ef28a-ad83-4c5f-a87d-e5703cde5813 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:08.794935 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ef28a-ad83-4c5f-a87d-e5703cde5813 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:08.800148 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 491/985] 10.222.0.57 () {56 vars in 1251 bytes} [Fri Sep 2 09:15:08 2022] GET /identity/v3/endpoints?service_id=2bea28b9432f4cfaaf9d3312ed015958&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:15:08.805351 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ef28a-ad83-4c5f-a87d-e5703cde5813 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:08.820603 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:08.827313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e9062-6efb-4792-83d7-26f0d1c50f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:08.827960 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e9062-6efb-4792-83d7-26f0d1c50f1c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:08.828248 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e9062-6efb-4792-83d7-26f0d1c50f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:08.828924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e9062-6efb-4792-83d7-26f0d1c50f1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:08.835820 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e9062-6efb-4792-83d7-26f0d1c50f1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:08.837262 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50e9062-6efb-4792-83d7-26f0d1c50f1c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:08.849465 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-f50e9062-6efb-4792-83d7-26f0d1c50f1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:15:08.849465 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:08.849465 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:08.849465 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:08.849465 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:08.849465 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:08.849465 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:08.849465 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:08.849465 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:08.849465 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:08.849465 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:08.849465 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:08.849465 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:15:08.849465 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:15:08.849465 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:15:08.850513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:15:08.850513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:08.850513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:08.850513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:08.850513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:08.850513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:08.850513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:08.850513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:08.850513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:08.850513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:08.850513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:08.850513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:08.850513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:08.850513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:08.850513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:08.850513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:08.850513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:08.851480 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:15:08.851480 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:15:08.851480 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:15:08.851480 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:15:08.851480 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:15:08.851480 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:15:08.851480 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:15:08.851480 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:08.851480 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 495/986] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:15:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:15:08.856201 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e9062-6efb-4792-83d7-26f0d1c50f1c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:08.904986 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:08.924606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ef28a-ad83-4c5f-a87d-e5703cde5813 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:08.959843 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858425a0-d630-4629-aa4e-7d2ea631625d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:08.960383 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858425a0-d630-4629-aa4e-7d2ea631625d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:08.960662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858425a0-d630-4629-aa4e-7d2ea631625d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:08.960941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858425a0-d630-4629-aa4e-7d2ea631625d None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:08.961549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858425a0-d630-4629-aa4e-7d2ea631625d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:08.962939 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858425a0-d630-4629-aa4e-7d2ea631625d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:08.963452 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50e9062-6efb-4792-83d7-26f0d1c50f1c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:08.969219 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 492/987] 10.222.0.57 () {56 vars in 1087 bytes} [Fri Sep 2 09:15:08 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:15:09.009909 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afbd19c-10a8-471f-b817-2afff6979064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:09.010523 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbd19c-10a8-471f-b817-2afff6979064 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:09.010837 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbd19c-10a8-471f-b817-2afff6979064 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:09.011106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afbd19c-10a8-471f-b817-2afff6979064 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:09.011791 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afbd19c-10a8-471f-b817-2afff6979064 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:09.012094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afbd19c-10a8-471f-b817-2afff6979064 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:09.013537 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afbd19c-10a8-471f-b817-2afff6979064 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:09.028268 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 496/988] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:15:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:09.034153 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858425a0-d630-4629-aa4e-7d2ea631625d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:09.145487 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858425a0-d630-4629-aa4e-7d2ea631625d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:09.178275 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a8ba6-f41d-4625-9256-5e8b375563f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:09.178922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a8ba6-f41d-4625-9256-5e8b375563f8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:09.179205 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a8ba6-f41d-4625-9256-5e8b375563f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:09.179479 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a8ba6-f41d-4625-9256-5e8b375563f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:09.180209 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a8ba6-f41d-4625-9256-5e8b375563f8 None admin] RBAC: Adding query filter params (role.id=dd2879946b3d4351acbaf8698b7966e8, scope.project.id=d18a5117a03b4f21ab3290a15680de66, user.id=5233689a53364258b86ad2fa012d8db1) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:09.180519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a8ba6-f41d-4625-9256-5e8b375563f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:09.181946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a8ba6-f41d-4625-9256-5e8b375563f8 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:09.187730 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 493/989] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:15:09 2022] GET /identity/v3/role_assignments?user.id=5233689a53364258b86ad2fa012d8db1&scope.project.id=d18a5117a03b4f21ab3290a15680de66&role.id=dd2879946b3d4351acbaf8698b7966e8 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:15:09.371699 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc44bb-9fb5-4982-962c-b5516b17bcc0 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:09.372215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc44bb-9fb5-4982-962c-b5516b17bcc0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:09.372302 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc44bb-9fb5-4982-962c-b5516b17bcc0 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:09.373597 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 497/990] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:09.379200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd76d93-7b01-4b14-88db-a68a301f8104 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:09.379480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd76d93-7b01-4b14-88db-a68a301f8104 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:09.379752 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd76d93-7b01-4b14-88db-a68a301f8104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:09.398075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-afd76d93-7b01-4b14-88db-a68a301f8104 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:09.399661 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-afd76d93-7b01-4b14-88db-a68a301f8104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:09.401860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:09.469091 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 494/991] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:09 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:15:09.735231 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc44bb-9fb5-4982-962c-b5516b17bcc0 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:09.788313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:09.838376 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc44bb-9fb5-4982-962c-b5516b17bcc0 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:09.881873 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2a91a-9b3c-466f-b738-e95ae93c9369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:09.882512 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2a91a-9b3c-466f-b738-e95ae93c9369 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:09.882819 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2a91a-9b3c-466f-b738-e95ae93c9369 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:09.883091 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2a91a-9b3c-466f-b738-e95ae93c9369 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:09.883668 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2a91a-9b3c-466f-b738-e95ae93c9369 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:09.885023 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef2a91a-9b3c-466f-b738-e95ae93c9369 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:09.893893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-3ef2a91a-9b3c-466f-b738-e95ae93c9369 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 09:15:09.893893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:09.893893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 09:15:09.893893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 09:15:09.893893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 09:15:09.893893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 09:15:09.893893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 09:15:09.893893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:15:09.893893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 09:15:09.893893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:09.893893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:15:09.893893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:09.893893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:09.893893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:09.893893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:09.893893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:09.893893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:09.895709 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:09.895709 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:09.895709 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:09.895709 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:09.895709 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:09.895709 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:09.895709 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 09:15:09.895709 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 09:15:09.895709 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 09:15:09.895709 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 09:15:09.895709 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:09.895709 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:09.895709 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:09.895709 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:09.895709 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:09.895709 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:09.896652 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:09.896652 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:09.896652 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:09.896652 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:09.896652 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:09.896652 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:09.896652 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:09.896652 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:09.896652 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:09.896652 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:09.896652 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 09:15:09.896652 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:15:09.896652 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 09:15:09.896652 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:09.896652 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 498/992] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Sep 2 09:15:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:09.903447 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd76d93-7b01-4b14-88db-a68a301f8104 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:10.003163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd76d93-7b01-4b14-88db-a68a301f8104 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:10.037260 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93538ce-0f2e-4611-b870-21314f7b978d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:10.037873 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93538ce-0f2e-4611-b870-21314f7b978d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:10.038154 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93538ce-0f2e-4611-b870-21314f7b978d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:10.038428 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93538ce-0f2e-4611-b870-21314f7b978d None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:10.039083 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93538ce-0f2e-4611-b870-21314f7b978d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:10.039390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93538ce-0f2e-4611-b870-21314f7b978d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:10.040740 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93538ce-0f2e-4611-b870-21314f7b978d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:10.045612 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 495/993] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Sep 2 09:15:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:10.051126 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2a91a-9b3c-466f-b738-e95ae93c9369 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:10.153043 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef2a91a-9b3c-466f-b738-e95ae93c9369 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:10.166951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:10.187847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b3667-e457-4bf9-8310-30f31d51502e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:10.188459 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b3667-e457-4bf9-8310-30f31d51502e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:10.188735 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b3667-e457-4bf9-8310-30f31d51502e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:10.189004 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b3667-e457-4bf9-8310-30f31d51502e None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:10.189635 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b3667-e457-4bf9-8310-30f31d51502e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:10.189937 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b3667-e457-4bf9-8310-30f31d51502e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:10.191331 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b3667-e457-4bf9-8310-30f31d51502e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:10.196810 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 499/994] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Sep 2 09:15:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:10.202516 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93538ce-0f2e-4611-b870-21314f7b978d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:10.205499 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:10.310584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93538ce-0f2e-4611-b870-21314f7b978d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:10.345680 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e840e-eabb-43e5-9cea-e9398459de61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:10.346356 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e840e-eabb-43e5-9cea-e9398459de61 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:10.346643 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e840e-eabb-43e5-9cea-e9398459de61 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:10.346944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e840e-eabb-43e5-9cea-e9398459de61 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:10.348288 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e840e-eabb-43e5-9cea-e9398459de61 None admin] RBAC: Adding query filter params (interface=admin, service_id=c9cb416f214a46ad9571659959580616, region_id=RegionOne) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:10.348606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e840e-eabb-43e5-9cea-e9398459de61 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:10.350018 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482e840e-eabb-43e5-9cea-e9398459de61 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:10.359637 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 496/995] 10.222.0.57 () {56 vars in 1249 bytes} [Fri Sep 2 09:15:10 2022] GET /identity/v3/endpoints?service_id=c9cb416f214a46ad9571659959580616&interface=admin®ion_id=RegionOne => generated 193 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:10.360578 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b3667-e457-4bf9-8310-30f31d51502e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:10.460597 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b3667-e457-4bf9-8310-30f31d51502e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:10.493916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24c066-3454-4db5-8b6b-61d55e04ddf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:10.494525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24c066-3454-4db5-8b6b-61d55e04ddf0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:10.494835 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24c066-3454-4db5-8b6b-61d55e04ddf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:10.495111 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24c066-3454-4db5-8b6b-61d55e04ddf0 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:10.495715 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d24c066-3454-4db5-8b6b-61d55e04ddf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:10.497121 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d24c066-3454-4db5-8b6b-61d55e04ddf0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:10.510758 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 500/996] 10.222.0.57 () {56 vars in 1087 bytes} [Fri Sep 2 09:15:10 2022] GET /identity/v3/services => generated 1727 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:15:10.902636 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c971802-f8d8-4c69-81a6-dcf9a29858fa None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:10.903203 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c971802-f8d8-4c69-81a6-dcf9a29858fa None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:10.903203 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c971802-f8d8-4c69-81a6-dcf9a29858fa None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:10.904703 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 497/997] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:10.909570 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907ba51-fd47-49be-9a17-c460a4a90c85 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:10.909856 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907ba51-fd47-49be-9a17-c460a4a90c85 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:10.911162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907ba51-fd47-49be-9a17-c460a4a90c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:10.929382 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1907ba51-fd47-49be-9a17-c460a4a90c85 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:10.930567 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1907ba51-fd47-49be-9a17-c460a4a90c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:10.968160 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:11.005073 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 501/998] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:15:11.140329 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999f163-c809-4779-bc4e-ff88401f70e0 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:11.140515 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999f163-c809-4779-bc4e-ff88401f70e0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:11.140796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999f163-c809-4779-bc4e-ff88401f70e0 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:11.142284 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 498/999] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:11.147314 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77b1e7-4eaa-4dda-8162-2a9ea9bab3c3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:11.147605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77b1e7-4eaa-4dda-8162-2a9ea9bab3c3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:11.147882 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77b1e7-4eaa-4dda-8162-2a9ea9bab3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:11.165694 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-bb77b1e7-4eaa-4dda-8162-2a9ea9bab3c3 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:11.166906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bb77b1e7-4eaa-4dda-8162-2a9ea9bab3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:11.232480 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 502/1000] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:11 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:15:11.271072 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999f163-c809-4779-bc4e-ff88401f70e0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:11.341447 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:11.377767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999f163-c809-4779-bc4e-ff88401f70e0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:11.409950 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1280d71-78d9-4af1-8d41-733913db0ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:11.411110 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1280d71-78d9-4af1-8d41-733913db0ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:11.411392 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1280d71-78d9-4af1-8d41-733913db0ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:11.411660 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1280d71-78d9-4af1-8d41-733913db0ac6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:11.412230 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1280d71-78d9-4af1-8d41-733913db0ac6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:11.413575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1280d71-78d9-4af1-8d41-733913db0ac6 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:11.422121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-c1280d71-78d9-4af1-8d41-733913db0ac6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 02 09:15:11.422121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:11.422121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 09:15:11.422121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 09:15:11.422121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 09:15:11.422121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 09:15:11.422121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 09:15:11.422121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:15:11.422121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 02 09:15:11.422121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:11.422121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:15:11.422121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:11.422121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:11.422121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:11.422121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:11.422121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:11.422121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:11.423380 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:11.423380 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:11.423380 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:11.423380 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:11.423380 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:11.423380 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:11.423380 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 09:15:11.423380 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 09:15:11.423380 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 09:15:11.423380 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 09:15:11.423380 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:11.423380 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:11.423380 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:11.423380 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:11.423380 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:11.423380 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:11.424258 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:11.424258 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:11.424258 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:11.424258 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:11.424258 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:11.424258 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:11.424258 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:11.424258 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:11.424258 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:11.424258 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:11.424258 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 09:15:11.424258 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:15:11.424258 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 02 09:15:11.424258 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:11.427025 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 499/1001] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:15:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:11.429644 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77b1e7-4eaa-4dda-8162-2a9ea9bab3c3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:11.504110 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1280d71-78d9-4af1-8d41-733913db0ac6 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:11.533741 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77b1e7-4eaa-4dda-8162-2a9ea9bab3c3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:11.535802 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:11.570909 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cf7f61-8f39-4612-b4b4-b145456b8152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:11.571701 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf7f61-8f39-4612-b4b4-b145456b8152 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:11.571986 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf7f61-8f39-4612-b4b4-b145456b8152 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:11.572267 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf7f61-8f39-4612-b4b4-b145456b8152 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:11.572918 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cf7f61-8f39-4612-b4b4-b145456b8152 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:11.573224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cf7f61-8f39-4612-b4b4-b145456b8152 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:11.574608 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cf7f61-8f39-4612-b4b4-b145456b8152 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:11.584357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 503/1002] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Sep 2 09:15:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:11.585410 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cf7f61-8f39-4612-b4b4-b145456b8152 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:11.620277 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1280d71-78d9-4af1-8d41-733913db0ac6 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:11.650865 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:11.656248 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1658c29-7e88-4de2-a365-d9d5647c8fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:11.656879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1658c29-7e88-4de2-a365-d9d5647c8fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:11.657149 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1658c29-7e88-4de2-a365-d9d5647c8fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:11.657428 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1658c29-7e88-4de2-a365-d9d5647c8fb2 None admin] PATH_INFO: `/v3/services/image` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:11.657996 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1658c29-7e88-4de2-a365-d9d5647c8fb2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:11.659518 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1658c29-7e88-4de2-a365-d9d5647c8fb2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:11.670821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-b1658c29-7e88-4de2-a365-d9d5647c8fb2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 02 09:15:11.670821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:11.670821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 09:15:11.670821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 09:15:11.670821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 09:15:11.670821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 09:15:11.670821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 09:15:11.670821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:15:11.670821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 02 09:15:11.670821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:11.670821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:15:11.670821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:11.670821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:11.670821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:11.670821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:11.670821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:11.670821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:11.671839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:11.671839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:11.671839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:11.671839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:11.671839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:11.671839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:11.671839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 09:15:11.671839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 09:15:11.671839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 09:15:11.671839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 09:15:11.671839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:11.671839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:11.671839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:11.671839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:11.671839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:11.671839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:11.672689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:11.672689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:11.672689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:11.672689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:11.672689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:11.672689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:11.672689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:11.672689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:11.672689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:11.672689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:11.672689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 09:15:11.672689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:15:11.672689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 02 09:15:11.672689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:11.672689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 500/1003] 10.222.0.57 () {56 vars in 1105 bytes} [Fri Sep 2 09:15:11 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:15:11.676917 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1658c29-7e88-4de2-a365-d9d5647c8fb2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:11.698510 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cf7f61-8f39-4612-b4b4-b145456b8152 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:11.735736 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceca850-4679-43d4-accc-0973350c7077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:11.736341 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceca850-4679-43d4-accc-0973350c7077 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:11.736632 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceca850-4679-43d4-accc-0973350c7077 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:11.736889 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceca850-4679-43d4-accc-0973350c7077 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:11.737517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceca850-4679-43d4-accc-0973350c7077 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:11.737820 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceca850-4679-43d4-accc-0973350c7077 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:11.740100 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceca850-4679-43d4-accc-0973350c7077 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:11.744875 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 504/1004] 10.222.0.57 () {56 vars in 1112 bytes} [Fri Sep 2 09:15:11 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:11.750325 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceca850-4679-43d4-accc-0973350c7077 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:11.780234 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:11.787314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1658c29-7e88-4de2-a365-d9d5647c8fb2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:11.822895 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c25462-d070-4e4b-b9e5-a20b9521fd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:11.823508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c25462-d070-4e4b-b9e5-a20b9521fd92 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:11.823789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c25462-d070-4e4b-b9e5-a20b9521fd92 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:11.824064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c25462-d070-4e4b-b9e5-a20b9521fd92 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:11.824698 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c25462-d070-4e4b-b9e5-a20b9521fd92 None admin] RBAC: Adding query filter params (name=image) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:11.825001 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c25462-d070-4e4b-b9e5-a20b9521fd92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:11.826353 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c25462-d070-4e4b-b9e5-a20b9521fd92 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:11.834744 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 501/1005] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:15:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:15:11.837194 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c25462-d070-4e4b-b9e5-a20b9521fd92 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:11.858052 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceca850-4679-43d4-accc-0973350c7077 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:11.895990 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ddc9c-91f9-42aa-b740-181f1d5053c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:11.896652 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ddc9c-91f9-42aa-b740-181f1d5053c6 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:11.896930 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ddc9c-91f9-42aa-b740-181f1d5053c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:11.897251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ddc9c-91f9-42aa-b740-181f1d5053c6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:11.897923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ddc9c-91f9-42aa-b740-181f1d5053c6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:11.899314 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ddc9c-91f9-42aa-b740-181f1d5053c6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:11.903908 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:11.946656 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c25462-d070-4e4b-b9e5-a20b9521fd92 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:11.980814 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a6096-b821-49a5-adb7-601b8f945543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:11.981425 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a6096-b821-49a5-adb7-601b8f945543 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:11.981701 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a6096-b821-49a5-adb7-601b8f945543 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:11.981974 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a6096-b821-49a5-adb7-601b8f945543 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:11.982599 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7a6096-b821-49a5-adb7-601b8f945543 None admin] RBAC: Adding query filter params (type=image) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:11.982926 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7a6096-b821-49a5-adb7-601b8f945543 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:11.984281 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7a6096-b821-49a5-adb7-601b8f945543 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:11.988996 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 502/1006] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:15:11 2022] GET /identity/v3/services?type=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:15:11.999237 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 505/1007] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:15:11 2022] POST /identity/v3/endpoints => generated 334 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:15:12.411140 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba264a81-5d42-42aa-b864-abd06f52be41 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:12.411539 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba264a81-5d42-42aa-b864-abd06f52be41 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:12.411735 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba264a81-5d42-42aa-b864-abd06f52be41 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:12.413204 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 503/1008] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:12.418130 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f5bc3-e7c7-4c68-bf61-d0855ae196ed None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:12.418419 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f5bc3-e7c7-4c68-bf61-d0855ae196ed None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:12.418693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109f5bc3-e7c7-4c68-bf61-d0855ae196ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:12.437249 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-109f5bc3-e7c7-4c68-bf61-d0855ae196ed None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:12.438435 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-109f5bc3-e7c7-4c68-bf61-d0855ae196ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:12.510625 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 506/1009] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:12 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:15:12.776844 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba264a81-5d42-42aa-b864-abd06f52be41 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:12.888092 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba264a81-5d42-42aa-b864-abd06f52be41 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:12.911865 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:12.923413 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf4220-fafc-4e23-a288-b726e92f8e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:12.924052 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf4220-fafc-4e23-a288-b726e92f8e79 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:12.924334 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf4220-fafc-4e23-a288-b726e92f8e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:12.927172 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cf4220-fafc-4e23-a288-b726e92f8e79 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:12.927172 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf4220-fafc-4e23-a288-b726e92f8e79 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:12.927326 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cf4220-fafc-4e23-a288-b726e92f8e79 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:12.936640 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-c1cf4220-fafc-4e23-a288-b726e92f8e79 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 09:15:12.936640 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:12.936640 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 09:15:12.936640 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 09:15:12.936640 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 09:15:12.936640 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 09:15:12.936640 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 09:15:12.936640 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:15:12.936640 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 09:15:12.936640 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:12.936640 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:15:12.936640 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:12.936640 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:12.936640 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:12.936640 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:12.936640 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:12.936640 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:12.937626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:12.937626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:12.937626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:12.937626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:12.937626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:12.937626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:12.937626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 09:15:12.937626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 09:15:12.937626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 09:15:12.937626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 09:15:12.937626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:12.937626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:12.937626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:12.937626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:12.937626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:12.937626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:12.943549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:12.943549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:12.943549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:12.943549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:12.943549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:12.943549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:12.943549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:12.943549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:12.943549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:12.943549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:12.943549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 09:15:12.943549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:15:12.943549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 02 09:15:12.943549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:12.943549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 504/1010] 10.222.0.57 () {56 vars in 1126 bytes} [Fri Sep 2 09:15:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:12.944367 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f5bc3-e7c7-4c68-bf61-d0855ae196ed None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:12.959708 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:13.045526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109f5bc3-e7c7-4c68-bf61-d0855ae196ed None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:13.081506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7713f9e5-e369-4520-a3f8-8c900b6b0d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:13.082151 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713f9e5-e369-4520-a3f8-8c900b6b0d56 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:13.082428 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713f9e5-e369-4520-a3f8-8c900b6b0d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:13.082697 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713f9e5-e369-4520-a3f8-8c900b6b0d56 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:13.083346 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7713f9e5-e369-4520-a3f8-8c900b6b0d56 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:13.083646 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7713f9e5-e369-4520-a3f8-8c900b6b0d56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:13.084988 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7713f9e5-e369-4520-a3f8-8c900b6b0d56 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:13.094315 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 507/1011] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Sep 2 09:15:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:13.095887 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf4220-fafc-4e23-a288-b726e92f8e79 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:13.199727 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cf4220-fafc-4e23-a288-b726e92f8e79 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:13.233082 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63fbba4-81e4-4368-a69e-607835821bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:13.233692 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63fbba4-81e4-4368-a69e-607835821bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:13.233973 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63fbba4-81e4-4368-a69e-607835821bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:13.234245 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63fbba4-81e4-4368-a69e-607835821bc0 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:13.234904 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63fbba4-81e4-4368-a69e-607835821bc0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:13.235212 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63fbba4-81e4-4368-a69e-607835821bc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:13.236575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63fbba4-81e4-4368-a69e-607835821bc0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:13.242027 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 505/1012] 10.222.0.57 () {56 vars in 1122 bytes} [Fri Sep 2 09:15:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:13.247783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7713f9e5-e369-4520-a3f8-8c900b6b0d56 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:13.324322 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:13.348201 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7713f9e5-e369-4520-a3f8-8c900b6b0d56 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:13.382420 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d49101-fa5f-480d-9edd-e78b544d7454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:13.383070 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d49101-fa5f-480d-9edd-e78b544d7454 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:13.383380 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d49101-fa5f-480d-9edd-e78b544d7454 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:13.383659 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d49101-fa5f-480d-9edd-e78b544d7454 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:13.384330 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d49101-fa5f-480d-9edd-e78b544d7454 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:13.385702 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d49101-fa5f-480d-9edd-e78b544d7454 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:13.450526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 508/1013] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:15:13 2022] POST /identity/v3/endpoints => generated 325 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:15:13.890199 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42c634-88d7-4ab7-a194-392d12b563ca None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:13.890537 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42c634-88d7-4ab7-a194-392d12b563ca None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:13.890779 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f42c634-88d7-4ab7-a194-392d12b563ca None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:13.892274 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 506/1014] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:13.897778 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478931e-b37f-489e-819d-7006745f7ba7 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:13.898070 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478931e-b37f-489e-819d-7006745f7ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:13.898339 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478931e-b37f-489e-819d-7006745f7ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:13.916182 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a478931e-b37f-489e-819d-7006745f7ba7 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:13.917369 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a478931e-b37f-489e-819d-7006745f7ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:13.990496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 509/1015] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:13 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:15:14.011032 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a60eb6-a4b3-41c6-8475-73e1fe6604a2 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:14.011498 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a60eb6-a4b3-41c6-8475-73e1fe6604a2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:14.011602 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a60eb6-a4b3-41c6-8475-73e1fe6604a2 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:14.013056 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 507/1016] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:14.017930 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff3fc8-ee38-4c2f-88d0-14e143838585 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:14.018225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff3fc8-ee38-4c2f-88d0-14e143838585 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:14.018503 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff3fc8-ee38-4c2f-88d0-14e143838585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:14.036278 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b4ff3fc8-ee38-4c2f-88d0-14e143838585 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:14.037454 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b4ff3fc8-ee38-4c2f-88d0-14e143838585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:14.092988 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:14.103697 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 510/1017] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:14 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:15:14.253281 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a60eb6-a4b3-41c6-8475-73e1fe6604a2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:14.304414 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:14.356453 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a60eb6-a4b3-41c6-8475-73e1fe6604a2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:14.372871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff3fc8-ee38-4c2f-88d0-14e143838585 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:14.391152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af207b1-3c94-4f11-b92d-9d6e841e7936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:14.391753 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af207b1-3c94-4f11-b92d-9d6e841e7936 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:14.392031 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af207b1-3c94-4f11-b92d-9d6e841e7936 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:14.392302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af207b1-3c94-4f11-b92d-9d6e841e7936 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:14.392944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af207b1-3c94-4f11-b92d-9d6e841e7936 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:14.394954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af207b1-3c94-4f11-b92d-9d6e841e7936 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:14.468896 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 508/1018] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Sep 2 09:15:14 2022] POST /identity/v3/services => generated 242 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:15:14.479390 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff3fc8-ee38-4c2f-88d0-14e143838585 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:14.527129 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651d226-0ca7-4ebd-9991-6cde07b723df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:14.528121 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651d226-0ca7-4ebd-9991-6cde07b723df None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:14.528577 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651d226-0ca7-4ebd-9991-6cde07b723df None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:14.529025 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651d226-0ca7-4ebd-9991-6cde07b723df None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:14.541392 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3651d226-0ca7-4ebd-9991-6cde07b723df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:15:14.546948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-3651d226-0ca7-4ebd-9991-6cde07b723df None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 09:15:14.546948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:14.546948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:14.546948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:14.546948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:14.546948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:14.546948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:14.546948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:14.546948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:14.546948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:14.546948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:14.546948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:14.546948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:15:14.546948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:15:14.546948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:15:14.546948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:15:14.547983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:15:14.547983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:15:14.547983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:15:14.547983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:15:14.547983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:14.547983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:14.547983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:14.547983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:14.547983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:14.547983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:14.547983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:14.547983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:14.547983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:14.547983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:14.547983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:14.547983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:14.548847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:14.548847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:14.548847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:14.548847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:14.548847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:15:14.548847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:15:14.548847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:15:14.548847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:15:14.548847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:15:14.548847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:15:14.548847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 09:15:14.548847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:14.551079 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 511/1019] 10.222.0.57 () {56 vars in 1120 bytes} [Fri Sep 2 09:15:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:15:14.557285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af207b1-3c94-4f11-b92d-9d6e841e7936 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:14.665181 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af207b1-3c94-4f11-b92d-9d6e841e7936 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:14.675462 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:14.699585 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b55052-a5cc-44e0-b26b-d698871d0ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:14.700194 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b55052-a5cc-44e0-b26b-d698871d0ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:14.700473 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b55052-a5cc-44e0-b26b-d698871d0ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:14.700792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b55052-a5cc-44e0-b26b-d698871d0ad5 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:14.701653 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b55052-a5cc-44e0-b26b-d698871d0ad5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:14.701964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b55052-a5cc-44e0-b26b-d698871d0ad5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:14.703519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b55052-a5cc-44e0-b26b-d698871d0ad5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:14.713667 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 509/1020] 10.222.0.57 () {56 vars in 1115 bytes} [Fri Sep 2 09:15:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:14.719592 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651d226-0ca7-4ebd-9991-6cde07b723df None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:14.768253 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:14.818255 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651d226-0ca7-4ebd-9991-6cde07b723df None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:14.857291 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9cc801-5cdd-4678-b9bd-74dccf0c19c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:14.857922 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cc801-5cdd-4678-b9bd-74dccf0c19c2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:14.858197 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cc801-5cdd-4678-b9bd-74dccf0c19c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:14.858466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cc801-5cdd-4678-b9bd-74dccf0c19c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:14.868970 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9cc801-5cdd-4678-b9bd-74dccf0c19c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:14.873621 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9cc801-5cdd-4678-b9bd-74dccf0c19c2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:14.890451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-5e9cc801-5cdd-4678-b9bd-74dccf0c19c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:15:14.890451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:14.890451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:14.890451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:14.890451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:14.890451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:14.890451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:14.890451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:14.890451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:14.890451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:14.890451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:14.890451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:14.890451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:15:14.890451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:15:14.890451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:15:14.890451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:15:14.891467 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:14.891467 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:14.891467 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:14.891467 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:14.891467 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:14.891467 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:14.891467 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:14.891467 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:14.891467 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:14.891467 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:14.891467 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:14.891467 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:14.891467 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:14.891467 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:14.891467 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:14.891467 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:14.891467 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:15:14.892366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:15:14.892366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:15:14.892366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:14.894774 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 512/1021] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:15:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:14.897450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b55052-a5cc-44e0-b26b-d698871d0ad5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:15.005678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b55052-a5cc-44e0-b26b-d698871d0ad5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:15.040269 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b45a2-06c7-4407-9fd9-fd91d0e22b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:15.041656 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b45a2-06c7-4407-9fd9-fd91d0e22b50 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:15.042142 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b45a2-06c7-4407-9fd9-fd91d0e22b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:15.042612 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b45a2-06c7-4407-9fd9-fd91d0e22b50 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:15.043469 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b45a2-06c7-4407-9fd9-fd91d0e22b50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:15.043966 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b45a2-06c7-4407-9fd9-fd91d0e22b50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:15.045552 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215b45a2-06c7-4407-9fd9-fd91d0e22b50 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:15.061998 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 510/1022] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:15:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:15.063892 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9cc801-5cdd-4678-b9bd-74dccf0c19c2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:15.162073 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:15.165603 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9cc801-5cdd-4678-b9bd-74dccf0c19c2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:15.199489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f468f-db17-4bd6-b0a7-d0de3936a1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:15.200110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f468f-db17-4bd6-b0a7-d0de3936a1d7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:15.200384 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f468f-db17-4bd6-b0a7-d0de3936a1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:15.200660 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f468f-db17-4bd6-b0a7-d0de3936a1d7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:15.208053 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f468f-db17-4bd6-b0a7-d0de3936a1d7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:15.209573 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f468f-db17-4bd6-b0a7-d0de3936a1d7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:15.220586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-ae1f468f-db17-4bd6-b0a7-d0de3936a1d7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 02 09:15:15.220586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:15.220586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:15.220586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:15.220586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:15.220586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:15.220586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:15.220586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:15.220586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:15.220586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:15.220586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:15.220586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:15.220586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:15:15.220586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:15:15.220586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:15:15.220586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:15:15.221550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:15.221550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:15.221550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:15:15.221550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:15:15.221550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:15:15.221550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:15:15.221550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:15.221550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:15.221550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:15.221550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:15.221550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:15.221550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:15.221550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:15.221550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:15.221550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:15.221550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:15.221550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:15.221550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:15.223059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:15.223059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:15.223059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:15:15.223059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:15:15.223059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:15:15.223059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:15:15.223059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:15:15.223059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:15:15.223059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 02 09:15:15.223059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:15.224359 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 513/1023] 10.222.0.57 () {56 vars in 1093 bytes} [Fri Sep 2 09:15:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:15.228057 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b45a2-06c7-4407-9fd9-fd91d0e22b50 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:15.260477 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:15.330315 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b45a2-06c7-4407-9fd9-fd91d0e22b50 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:15.365128 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8d5dbb-74b3-4247-8cdf-515326872359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:15.365740 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d5dbb-74b3-4247-8cdf-515326872359 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:15.366018 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d5dbb-74b3-4247-8cdf-515326872359 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:15.366288 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8d5dbb-74b3-4247-8cdf-515326872359 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:15.367043 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8d5dbb-74b3-4247-8cdf-515326872359 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:15.367350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8d5dbb-74b3-4247-8cdf-515326872359 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:15.368761 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8d5dbb-74b3-4247-8cdf-515326872359 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:15.390767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 511/1024] 10.222.0.57 () {56 vars in 1133 bytes} [Fri Sep 2 09:15:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:15.391905 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f468f-db17-4bd6-b0a7-d0de3936a1d7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:15.396987 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c593b-223e-4314-a957-edb670f4e67b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:15.397270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c593b-223e-4314-a957-edb670f4e67b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:15.397542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85c593b-223e-4314-a957-edb670f4e67b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:15.399007 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 512/1025] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:15:15.404047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e13bf-5635-4ca6-983f-e67634fdae21 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:15.404327 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e13bf-5635-4ca6-983f-e67634fdae21 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:15.404715 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92e13bf-5635-4ca6-983f-e67634fdae21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:15.422796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b92e13bf-5635-4ca6-983f-e67634fdae21 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:15.423972 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b92e13bf-5635-4ca6-983f-e67634fdae21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:15.496642 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 513/1026] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 09:15:15.507548 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f468f-db17-4bd6-b0a7-d0de3936a1d7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:15.541233 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec99a53-7af3-4e05-8005-c5618b20c144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:15.541768 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec99a53-7af3-4e05-8005-c5618b20c144 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:15.542049 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec99a53-7af3-4e05-8005-c5618b20c144 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:15.542322 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec99a53-7af3-4e05-8005-c5618b20c144 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:15.572016 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:15.614915 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec99a53-7af3-4e05-8005-c5618b20c144 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:15.616360 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec99a53-7af3-4e05-8005-c5618b20c144 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:15.713900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 514/1027] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:15:15 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 09:15:15.719600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92e13bf-5635-4ca6-983f-e67634fdae21 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:15.741798 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:15.766616 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec99a53-7af3-4e05-8005-c5618b20c144 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:15.823280 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92e13bf-5635-4ca6-983f-e67634fdae21 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:15.830626 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:15.867878 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05fabf5-a9cb-4e16-809d-4da9d0dac3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:15.868762 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05fabf5-a9cb-4e16-809d-4da9d0dac3d2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:15.868762 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05fabf5-a9cb-4e16-809d-4da9d0dac3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:15.869067 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05fabf5-a9cb-4e16-809d-4da9d0dac3d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:15.881366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec99a53-7af3-4e05-8005-c5618b20c144 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:15.882905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05fabf5-a9cb-4e16-809d-4da9d0dac3d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:15.884358 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05fabf5-a9cb-4e16-809d-4da9d0dac3d2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:15.899473 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-a05fabf5-a9cb-4e16-809d-4da9d0dac3d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:15:15.899473 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:15.899473 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:15.899473 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:15.899473 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:15.899473 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:15.899473 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:15.899473 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:15.899473 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:15.899473 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:15.899473 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:15.899473 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:15.899473 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:15:15.899473 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:15:15.899473 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:15:15.899473 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:15:15.900434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:15.900434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:15.900434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:15.900434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:15.900434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:15.900434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:15.900434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:15.900434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:15.900434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:15.900434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:15.900434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:15.900434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:15.900434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:15.900434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:15.900434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:15.900434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:15.900434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:15:15.901344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:15:15.901344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:15:15.901344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:15.901344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 514/1028] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:15:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:15.906068 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05fabf5-a9cb-4e16-809d-4da9d0dac3d2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:15.918857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6a4fc-8f1d-405b-9fe6-6c01c0433b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:15.919483 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6a4fc-8f1d-405b-9fe6-6c01c0433b7c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:15.919761 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6a4fc-8f1d-405b-9fe6-6c01c0433b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:15.920027 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6a4fc-8f1d-405b-9fe6-6c01c0433b7c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:15.921106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af6a4fc-8f1d-405b-9fe6-6c01c0433b7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:15.922511 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af6a4fc-8f1d-405b-9fe6-6c01c0433b7c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:15.926884 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 515/1029] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:15:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:15:15.929809 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6a4fc-8f1d-405b-9fe6-6c01c0433b7c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:15.982556 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:16.015423 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05fabf5-a9cb-4e16-809d-4da9d0dac3d2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:16.038635 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6a4fc-8f1d-405b-9fe6-6c01c0433b7c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:16.051958 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46bc06-6f6f-44e7-aeae-3270cbe388fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:16.052571 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46bc06-6f6f-44e7-aeae-3270cbe388fd None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:16.052855 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46bc06-6f6f-44e7-aeae-3270cbe388fd None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:16.053129 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46bc06-6f6f-44e7-aeae-3270cbe388fd None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:16.053770 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46bc06-6f6f-44e7-aeae-3270cbe388fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:16.054073 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46bc06-6f6f-44e7-aeae-3270cbe388fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:16.055449 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46bc06-6f6f-44e7-aeae-3270cbe388fd None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:16.067078 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:16.069582 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 515/1030] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:15:16.080653 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46bc06-6f6f-44e7-aeae-3270cbe388fd None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:16.080826 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffda39e-fbb5-4d50-9fef-9e7772ffb705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:16.080826 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffda39e-fbb5-4d50-9fef-9e7772ffb705 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:16.080826 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffda39e-fbb5-4d50-9fef-9e7772ffb705 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:16.080826 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffda39e-fbb5-4d50-9fef-9e7772ffb705 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:16.081206 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffda39e-fbb5-4d50-9fef-9e7772ffb705 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:16.081206 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffda39e-fbb5-4d50-9fef-9e7772ffb705 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:16.174382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46bc06-6f6f-44e7-aeae-3270cbe388fd None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:16.189682 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 516/1031] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Sep 2 09:15:15 2022] POST /identity/v3/projects => generated 311 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:15:16.215464 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:16.221401 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83317ca-f127-440b-89e2-7f0bdd71c909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:16.222069 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83317ca-f127-440b-89e2-7f0bdd71c909 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:16.222377 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83317ca-f127-440b-89e2-7f0bdd71c909 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:16.222652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83317ca-f127-440b-89e2-7f0bdd71c909 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:16.230546 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83317ca-f127-440b-89e2-7f0bdd71c909 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:16.232055 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83317ca-f127-440b-89e2-7f0bdd71c909 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:16.245337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-d83317ca-f127-440b-89e2-7f0bdd71c909 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:15:16.245337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:16.245337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:16.245337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:16.245337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:16.245337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:16.245337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:16.245337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:16.245337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:16.245337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:16.245337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:16.245337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:16.245337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:15:16.245337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:15:16.245337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:15:16.246355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:15:16.246355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:16.246355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:16.246355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:16.246355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:16.246355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:16.246355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:16.246355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:16.246355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:16.246355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:16.246355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:16.246355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:16.246355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:16.246355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:16.246355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:16.246355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:16.246355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:16.252925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:15:16.252925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:15:16.252925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:15:16.252925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:15:16.252925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:15:16.252925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:15:16.252925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:15:16.252925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:16.252925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 516/1032] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:15:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:15:16.254139 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffda39e-fbb5-4d50-9fef-9e7772ffb705 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:16.362283 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffda39e-fbb5-4d50-9fef-9e7772ffb705 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:16.398194 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e36631-207e-4c19-8cce-62ea606d8ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:16.398836 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e36631-207e-4c19-8cce-62ea606d8ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:16.399122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e36631-207e-4c19-8cce-62ea606d8ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:16.399397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e36631-207e-4c19-8cce-62ea606d8ca7 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:16.400112 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e36631-207e-4c19-8cce-62ea606d8ca7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:16.400421 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e36631-207e-4c19-8cce-62ea606d8ca7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:16.401844 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e36631-207e-4c19-8cce-62ea606d8ca7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:16.414248 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 517/1033] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:15:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:16.420399 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83317ca-f127-440b-89e2-7f0bdd71c909 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:16.444565 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580b0bf-51eb-4121-b6a5-b21784c7d1e4 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:16.444856 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580b0bf-51eb-4121-b6a5-b21784c7d1e4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:16.445134 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580b0bf-51eb-4121-b6a5-b21784c7d1e4 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:16.446957 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 518/1034] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:15:16.452734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d5500-6d92-4d2a-9f18-c24ee912eb01 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:16.453094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d5500-6d92-4d2a-9f18-c24ee912eb01 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:16.453440 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d5500-6d92-4d2a-9f18-c24ee912eb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:16.471375 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-692d5500-6d92-4d2a-9f18-c24ee912eb01 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:16.472817 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-692d5500-6d92-4d2a-9f18-c24ee912eb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:16.510037 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:16.531872 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83317ca-f127-440b-89e2-7f0bdd71c909 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:16.544677 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 519/1035] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 09:15:16.571246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1842c-48b9-4dad-9c5a-bfdc7514ca29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:16.571861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1842c-48b9-4dad-9c5a-bfdc7514ca29 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:16.572145 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1842c-48b9-4dad-9c5a-bfdc7514ca29 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:16.572420 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1842c-48b9-4dad-9c5a-bfdc7514ca29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:16.573150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1842c-48b9-4dad-9c5a-bfdc7514ca29 None admin] RBAC: Adding query filter params (role.id=dd2879946b3d4351acbaf8698b7966e8, scope.project.id=d18a5117a03b4f21ab3290a15680de66, user.id=1da355c8be2d496bb08feb0fe74e04b9) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:16.573461 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1842c-48b9-4dad-9c5a-bfdc7514ca29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:16.574905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1842c-48b9-4dad-9c5a-bfdc7514ca29 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:16.580722 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 517/1036] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:15:16 2022] GET /identity/v3/role_assignments?user.id=1da355c8be2d496bb08feb0fe74e04b9&scope.project.id=d18a5117a03b4f21ab3290a15680de66&role.id=dd2879946b3d4351acbaf8698b7966e8 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:16.815499 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d5500-6d92-4d2a-9f18-c24ee912eb01 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:16.921208 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d5500-6d92-4d2a-9f18-c24ee912eb01 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:16.955874 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b096fc-00ac-4f04-ae3e-e818429f59c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:16.956539 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b096fc-00ac-4f04-ae3e-e818429f59c7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:16.956822 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b096fc-00ac-4f04-ae3e-e818429f59c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:16.957097 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b096fc-00ac-4f04-ae3e-e818429f59c7 None admin] PATH_INFO: `/v3/services/image` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:16.957676 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b096fc-00ac-4f04-ae3e-e818429f59c7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:16.959068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b096fc-00ac-4f04-ae3e-e818429f59c7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:16.967924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-a6b096fc-00ac-4f04-ae3e-e818429f59c7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 02 09:15:16.967924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:16.967924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 09:15:16.967924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 09:15:16.967924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 09:15:16.967924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 09:15:16.967924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 09:15:16.967924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:15:16.967924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 02 09:15:16.967924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:16.967924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:15:16.967924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:16.967924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:16.967924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:16.967924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:16.967924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:16.967924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:16.968921 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:16.968921 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:16.968921 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:16.968921 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:16.968921 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:16.968921 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:16.968921 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 09:15:16.968921 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 09:15:16.968921 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 09:15:16.968921 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 09:15:16.968921 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:16.968921 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:16.968921 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:16.968921 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:16.968921 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:16.968921 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:16.969765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:16.969765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:16.969765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:16.969765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:16.969765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:16.969765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:16.969765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:16.969765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:16.969765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:16.969765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:16.969765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 09:15:16.969765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:15:16.969765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 02 09:15:16.969765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:16.974465 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 520/1037] 10.222.0.57 () {56 vars in 1105 bytes} [Fri Sep 2 09:15:16 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:16.975760 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1842c-48b9-4dad-9c5a-bfdc7514ca29 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:17.041061 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:17.084568 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1842c-48b9-4dad-9c5a-bfdc7514ca29 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:17.119345 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a4a3d-2eaf-44f8-b651-ab946946964f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:17.119949 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a4a3d-2eaf-44f8-b651-ab946946964f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:17.120227 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a4a3d-2eaf-44f8-b651-ab946946964f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:17.120498 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a4a3d-2eaf-44f8-b651-ab946946964f None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:17.121117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a4a3d-2eaf-44f8-b651-ab946946964f None admin] RBAC: Adding query filter params (name=image) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:17.121419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a4a3d-2eaf-44f8-b651-ab946946964f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:17.122841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a4a3d-2eaf-44f8-b651-ab946946964f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:17.132046 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 518/1038] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:15:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:17.133283 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b096fc-00ac-4f04-ae3e-e818429f59c7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:17.169529 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:17.236505 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b096fc-00ac-4f04-ae3e-e818429f59c7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:17.290254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d508bc-4030-411c-92dd-6e0a9ec8bba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:17.291214 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d508bc-4030-411c-92dd-6e0a9ec8bba3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:17.291682 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d508bc-4030-411c-92dd-6e0a9ec8bba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:17.292160 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d508bc-4030-411c-92dd-6e0a9ec8bba3 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:17.293289 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d508bc-4030-411c-92dd-6e0a9ec8bba3 None admin] RBAC: Adding query filter params (type=image) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:17.293815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d508bc-4030-411c-92dd-6e0a9ec8bba3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:17.296341 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d508bc-4030-411c-92dd-6e0a9ec8bba3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:17.304349 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 521/1039] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:15:17 2022] GET /identity/v3/services?type=image => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:17.310086 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a4a3d-2eaf-44f8-b651-ab946946964f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:17.418000 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a4a3d-2eaf-44f8-b651-ab946946964f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:17.437172 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:17.451988 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01c232-97ff-41e0-9952-bb9a67db6dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:17.452600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01c232-97ff-41e0-9952-bb9a67db6dba None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:17.452881 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01c232-97ff-41e0-9952-bb9a67db6dba None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:17.453154 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01c232-97ff-41e0-9952-bb9a67db6dba None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:17.453852 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e01c232-97ff-41e0-9952-bb9a67db6dba None admin] RBAC: Adding query filter params (interface=public, service_id=fe0412e5a25e44f4a5f55dd77d1611aa, region_id=RegionOne) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:17.454160 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e01c232-97ff-41e0-9952-bb9a67db6dba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:17.455542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e01c232-97ff-41e0-9952-bb9a67db6dba None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:17.461221 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 519/1040] 10.222.0.57 () {56 vars in 1251 bytes} [Fri Sep 2 09:15:17 2022] GET /identity/v3/endpoints?service_id=fe0412e5a25e44f4a5f55dd77d1611aa&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:15:17.466866 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d508bc-4030-411c-92dd-6e0a9ec8bba3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:17.564147 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:17.568627 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d508bc-4030-411c-92dd-6e0a9ec8bba3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:17.604575 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f800874-8c1d-4a7f-8dbf-a5400f75850e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:17.605513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f800874-8c1d-4a7f-8dbf-a5400f75850e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:17.605952 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f800874-8c1d-4a7f-8dbf-a5400f75850e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:17.606386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f800874-8c1d-4a7f-8dbf-a5400f75850e None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:17.607286 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f800874-8c1d-4a7f-8dbf-a5400f75850e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:17.608649 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f800874-8c1d-4a7f-8dbf-a5400f75850e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:17.617268 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 522/1041] 10.222.0.57 () {56 vars in 1087 bytes} [Fri Sep 2 09:15:17 2022] GET /identity/v3/services => generated 1957 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:15:18.229080 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647bd2b-e4f3-4b6c-b4dd-7c99765cd624 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:18.229567 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647bd2b-e4f3-4b6c-b4dd-7c99765cd624 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:18.229567 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647bd2b-e4f3-4b6c-b4dd-7c99765cd624 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:18.230967 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 520/1042] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:18.236022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4638417-913e-4220-8c17-a00e8c21fac6 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:18.236401 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4638417-913e-4220-8c17-a00e8c21fac6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:18.236584 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4638417-913e-4220-8c17-a00e8c21fac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:18.255298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c4638417-913e-4220-8c17-a00e8c21fac6 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:18.256562 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c4638417-913e-4220-8c17-a00e8c21fac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:18.326790 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 523/1043] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:15:18.453190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722d264-cb27-40e3-81f0-5678322778f9 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:18.453585 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722d264-cb27-40e3-81f0-5678322778f9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:18.453771 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722d264-cb27-40e3-81f0-5678322778f9 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:18.455395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 521/1044] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:18.461116 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67248827-3030-423b-8ea3-a5928cbbca03 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:18.461410 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67248827-3030-423b-8ea3-a5928cbbca03 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:18.461690 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67248827-3030-423b-8ea3-a5928cbbca03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:18.480312 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-67248827-3030-423b-8ea3-a5928cbbca03 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:18.481449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-67248827-3030-423b-8ea3-a5928cbbca03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:18.549897 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 524/1045] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:15:18.590280 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722d264-cb27-40e3-81f0-5678322778f9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:18.698840 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722d264-cb27-40e3-81f0-5678322778f9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:18.733615 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf20aa-ad59-4ad9-84d7-fb242a8ba791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:18.734159 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf20aa-ad59-4ad9-84d7-fb242a8ba791 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:18.734443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf20aa-ad59-4ad9-84d7-fb242a8ba791 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:18.734726 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf20aa-ad59-4ad9-84d7-fb242a8ba791 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:18.735850 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf20aa-ad59-4ad9-84d7-fb242a8ba791 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:18.737244 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf20aa-ad59-4ad9-84d7-fb242a8ba791 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:18.739028 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 522/1046] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:15:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:18.744534 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67248827-3030-423b-8ea3-a5928cbbca03 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:18.769506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:18.819724 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf20aa-ad59-4ad9-84d7-fb242a8ba791 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:18.851067 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67248827-3030-423b-8ea3-a5928cbbca03 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:18.882965 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:18.885405 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3074dda-5505-4137-b6d8-104ceb901a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:18.886011 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3074dda-5505-4137-b6d8-104ceb901a80 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:18.886287 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3074dda-5505-4137-b6d8-104ceb901a80 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:18.886591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3074dda-5505-4137-b6d8-104ceb901a80 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:18.887285 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3074dda-5505-4137-b6d8-104ceb901a80 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:18.888695 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3074dda-5505-4137-b6d8-104ceb901a80 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:18.931293 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf20aa-ad59-4ad9-84d7-fb242a8ba791 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:18.965948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8526f6-14ca-4519-99f8-e0fdbf323447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:18.966582 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8526f6-14ca-4519-99f8-e0fdbf323447 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:18.966893 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8526f6-14ca-4519-99f8-e0fdbf323447 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:18.967164 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8526f6-14ca-4519-99f8-e0fdbf323447 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:18.977189 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8526f6-14ca-4519-99f8-e0fdbf323447 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:15:18.982325 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-9a8526f6-14ca-4519-99f8-e0fdbf323447 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 09:15:18.982325 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:18.982325 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:18.982325 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:18.982325 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:18.982325 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:18.982325 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:18.982325 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:18.982325 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:18.982325 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:18.982325 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:18.982325 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:18.982325 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:15:18.982325 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:15:18.982325 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:15:18.982325 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:15:18.983600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:15:18.983600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:15:18.983600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:15:18.983600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:15:18.983600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:18.983600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:18.983600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:18.983600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:18.983600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:18.983600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:18.983600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:18.983600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:18.983600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:18.983600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:18.983600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:18.983600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:18.984543 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:18.984543 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:18.984543 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:18.984543 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:18.984543 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:15:18.984543 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:15:18.984543 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:15:18.984543 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:15:18.984543 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:15:18.984543 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:15:18.984543 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 09:15:18.984543 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:18.984543 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 523/1047] 10.222.0.57 () {56 vars in 1120 bytes} [Fri Sep 2 09:15:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:15:18.989354 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8526f6-14ca-4519-99f8-e0fdbf323447 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:19.020223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 525/1048] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Sep 2 09:15:18 2022] POST /identity/v3/users => generated 285 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:15:19.107549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8526f6-14ca-4519-99f8-e0fdbf323447 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:19.121542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:19.145423 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae9dfb-6ef7-49bb-8261-c4a8c431c5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:19.146052 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae9dfb-6ef7-49bb-8261-c4a8c431c5da None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:19.146329 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae9dfb-6ef7-49bb-8261-c4a8c431c5da None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:19.146596 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae9dfb-6ef7-49bb-8261-c4a8c431c5da None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:19.147360 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae9dfb-6ef7-49bb-8261-c4a8c431c5da None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:19.147878 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae9dfb-6ef7-49bb-8261-c4a8c431c5da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:19.149748 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae9dfb-6ef7-49bb-8261-c4a8c431c5da None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:19.158454 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 524/1049] 10.222.0.57 () {56 vars in 1115 bytes} [Fri Sep 2 09:15:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:15:19.164406 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3074dda-5505-4137-b6d8-104ceb901a80 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:19.248679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:19.270119 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3074dda-5505-4137-b6d8-104ceb901a80 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:19.304007 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8ac23-9cb0-403d-820d-d437fb6eaee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:19.304700 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8ac23-9cb0-403d-820d-d437fb6eaee2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:19.304981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8ac23-9cb0-403d-820d-d437fb6eaee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:19.305256 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8ac23-9cb0-403d-820d-d437fb6eaee2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:19.317062 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa8ac23-9cb0-403d-820d-d437fb6eaee2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:19.318729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa8ac23-9cb0-403d-820d-d437fb6eaee2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:19.334927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-bfa8ac23-9cb0-403d-820d-d437fb6eaee2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:15:19.334927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:19.334927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:19.334927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:19.334927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:19.334927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:19.334927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:19.334927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:19.334927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:19.334927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:19.334927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:19.334927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:19.334927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:15:19.334927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:15:19.334927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:15:19.334927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:15:19.335890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:19.335890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:19.335890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:19.335890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:19.335890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:19.335890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:19.335890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:19.335890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:19.335890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:19.335890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:19.335890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:19.335890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:19.335890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:19.335890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:19.335890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:19.335890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:19.335890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:15:19.336791 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:15:19.336791 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:15:19.336791 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:19.336791 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 526/1050] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:15:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:15:19.342746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae9dfb-6ef7-49bb-8261-c4a8c431c5da None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:19.446020 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae9dfb-6ef7-49bb-8261-c4a8c431c5da None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:19.478789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5204c7-5625-4483-b9a0-f0127ce75383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:19.479398 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5204c7-5625-4483-b9a0-f0127ce75383 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:19.479678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5204c7-5625-4483-b9a0-f0127ce75383 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:19.479950 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5204c7-5625-4483-b9a0-f0127ce75383 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:19.480592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5204c7-5625-4483-b9a0-f0127ce75383 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:19.480894 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5204c7-5625-4483-b9a0-f0127ce75383 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:19.482248 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5204c7-5625-4483-b9a0-f0127ce75383 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:19.499119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 525/1051] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:19.500669 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8ac23-9cb0-403d-820d-d437fb6eaee2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:19.544712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce31e6-55b8-4c5e-85a7-b36e35663c61 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:19.544991 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce31e6-55b8-4c5e-85a7-b36e35663c61 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:19.545263 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce31e6-55b8-4c5e-85a7-b36e35663c61 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:19.546634 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 526/1052] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:15:19.552061 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7614a-fef2-497a-b367-9eb2c47d7a99 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:19.552427 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7614a-fef2-497a-b367-9eb2c47d7a99 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:19.552705 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7614a-fef2-497a-b367-9eb2c47d7a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:19.571921 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7ff7614a-fef2-497a-b367-9eb2c47d7a99 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:19.573209 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7ff7614a-fef2-497a-b367-9eb2c47d7a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:19.600974 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:19.608060 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8ac23-9cb0-403d-820d-d437fb6eaee2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:19.636854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:19.643301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662ebf3-9cfb-41ee-b592-fd0688ad5b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:19.643967 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662ebf3-9cfb-41ee-b592-fd0688ad5b5d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:19.644241 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662ebf3-9cfb-41ee-b592-fd0688ad5b5d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:19.650989 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 527/1053] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 09:15:19.651088 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a662ebf3-9cfb-41ee-b592-fd0688ad5b5d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:19.652183 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662ebf3-9cfb-41ee-b592-fd0688ad5b5d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:19.653624 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a662ebf3-9cfb-41ee-b592-fd0688ad5b5d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:19.665075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-a662ebf3-9cfb-41ee-b592-fd0688ad5b5d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 02 09:15:19.665075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:19.665075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:19.665075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:19.665075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:19.665075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:19.665075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:19.665075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:19.665075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:19.665075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:19.665075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:19.665075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:19.665075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:15:19.665075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:15:19.665075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:15:19.665075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:15:19.666041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:19.666041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:19.666041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:15:19.666041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:15:19.666041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:15:19.666041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:15:19.666041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:19.666041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:19.666041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:19.666041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:19.666041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:19.666041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:19.666041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:19.666041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:19.666041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:19.666041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:19.666041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:19.666041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:19.667065 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:19.667065 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:19.667065 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:15:19.667065 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:15:19.667065 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:15:19.667065 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:15:19.667065 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:15:19.667065 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:15:19.667065 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 02 09:15:19.667065 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:19.670872 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 527/1054] 10.222.0.57 () {56 vars in 1093 bytes} [Fri Sep 2 09:15:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:15:19.672118 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff7614a-fef2-497a-b367-9eb2c47d7a99 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:19.772768 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff7614a-fef2-497a-b367-9eb2c47d7a99 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:19.807405 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc27cf3-2893-49d9-8fdd-9f45e7aa6136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:19.808004 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc27cf3-2893-49d9-8fdd-9f45e7aa6136 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:19.808284 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc27cf3-2893-49d9-8fdd-9f45e7aa6136 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:19.808553 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc27cf3-2893-49d9-8fdd-9f45e7aa6136 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:19.809264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc27cf3-2893-49d9-8fdd-9f45e7aa6136 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:19.809567 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc27cf3-2893-49d9-8fdd-9f45e7aa6136 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:19.810996 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc27cf3-2893-49d9-8fdd-9f45e7aa6136 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:19.827899 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 528/1055] 10.222.0.57 () {56 vars in 1133 bytes} [Fri Sep 2 09:15:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:19.833311 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662ebf3-9cfb-41ee-b592-fd0688ad5b5d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:19.923592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc27cf3-2893-49d9-8fdd-9f45e7aa6136 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:19.936247 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a662ebf3-9cfb-41ee-b592-fd0688ad5b5d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:19.949053 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:19.971255 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c0ca0-d6ca-41ca-9c03-dfce59ba4880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:19.972403 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c0ca0-d6ca-41ca-9c03-dfce59ba4880 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:19.972691 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c0ca0-d6ca-41ca-9c03-dfce59ba4880 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:19.972961 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c0ca0-d6ca-41ca-9c03-dfce59ba4880 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:19.983867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c0ca0-d6ca-41ca-9c03-dfce59ba4880 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:19.985289 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c0ca0-d6ca-41ca-9c03-dfce59ba4880 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:20.000094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-164c0ca0-d6ca-41ca-9c03-dfce59ba4880 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:15:20.000094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:20.000094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:20.000094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:20.000094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:20.000094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:20.000094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:20.000094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:20.000094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:20.000094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:20.000094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:20.000094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:20.000094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:15:20.000094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:15:20.000094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:15:20.000094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:15:20.001251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:20.001251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:20.001251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:20.001251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:20.001251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:20.001251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:20.001251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:20.001251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:20.001251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:20.001251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:20.001251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:20.001251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:20.001251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:20.001251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:20.001251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:20.001251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:20.001251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:15:20.002136 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:15:20.002136 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:15:20.002136 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:20.002136 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 528/1056] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:15:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:20.006814 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c0ca0-d6ca-41ca-9c03-dfce59ba4880 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:20.031587 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc27cf3-2893-49d9-8fdd-9f45e7aa6136 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:20.051136 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:20.068063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a56603-0da6-4b93-b29b-55735d4aaee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:20.068991 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a56603-0da6-4b93-b29b-55735d4aaee5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:20.069462 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a56603-0da6-4b93-b29b-55735d4aaee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:20.069908 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a56603-0da6-4b93-b29b-55735d4aaee5 None admin] PATH_INFO: `/v3/services/image` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:20.070945 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a56603-0da6-4b93-b29b-55735d4aaee5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:20.072584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a56603-0da6-4b93-b29b-55735d4aaee5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:20.082241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-d1a56603-0da6-4b93-b29b-55735d4aaee5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 02 09:15:20.082241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:20.082241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 09:15:20.082241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 09:15:20.082241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 09:15:20.082241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 09:15:20.082241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 09:15:20.082241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:15:20.082241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 02 09:15:20.082241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:20.082241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:15:20.082241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:20.082241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:20.082241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:20.082241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:20.082241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:20.082241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:20.083419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:20.083419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:20.083419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:20.083419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:20.083419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:20.083419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:20.083419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 09:15:20.083419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 09:15:20.083419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 09:15:20.083419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 09:15:20.083419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:20.083419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:20.083419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:20.083419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:20.083419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:20.083419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:20.084465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:20.084465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:20.084465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:20.084465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:20.084465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:20.084465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:20.084465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:20.084465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:20.084465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:20.084465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:20.084465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 09:15:20.084465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:15:20.084465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 02 09:15:20.084465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:20.086535 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 529/1057] 10.222.0.57 () {56 vars in 1105 bytes} [Fri Sep 2 09:15:19 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:15:20.092257 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a56603-0da6-4b93-b29b-55735d4aaee5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:20.116486 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c0ca0-d6ca-41ca-9c03-dfce59ba4880 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:20.150859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0d560-ee98-41e7-adc1-6af37dbd5577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:20.151705 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0d560-ee98-41e7-adc1-6af37dbd5577 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:20.151984 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0d560-ee98-41e7-adc1-6af37dbd5577 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:20.152259 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0d560-ee98-41e7-adc1-6af37dbd5577 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:20.152906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0d560-ee98-41e7-adc1-6af37dbd5577 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:20.153208 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0d560-ee98-41e7-adc1-6af37dbd5577 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:20.154565 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0d560-ee98-41e7-adc1-6af37dbd5577 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:20.167851 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 529/1058] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:20.173673 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0d560-ee98-41e7-adc1-6af37dbd5577 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:20.196361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:20.211636 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a56603-0da6-4b93-b29b-55735d4aaee5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:20.246836 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6c167-f906-47d9-aab9-1efef09d03b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:20.247555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6c167-f906-47d9-aab9-1efef09d03b2 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:20.247839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6c167-f906-47d9-aab9-1efef09d03b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:20.248112 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6c167-f906-47d9-aab9-1efef09d03b2 None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:20.248739 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6c167-f906-47d9-aab9-1efef09d03b2 None admin] RBAC: Adding query filter params (name=image) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:20.249044 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6c167-f906-47d9-aab9-1efef09d03b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:20.250433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6c167-f906-47d9-aab9-1efef09d03b2 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:20.255560 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 530/1059] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:15:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:15:20.261198 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6c167-f906-47d9-aab9-1efef09d03b2 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:20.285269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0d560-ee98-41e7-adc1-6af37dbd5577 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:20.286886 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:20.321792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e59fd-c2e2-44a2-839e-d4500dee0dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:20.322497 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e59fd-c2e2-44a2-839e-d4500dee0dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:20.322806 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e59fd-c2e2-44a2-839e-d4500dee0dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:20.323161 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e59fd-c2e2-44a2-839e-d4500dee0dd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:20.331654 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586e59fd-c2e2-44a2-839e-d4500dee0dd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:20.333185 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586e59fd-c2e2-44a2-839e-d4500dee0dd3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:20.345110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-586e59fd-c2e2-44a2-839e-d4500dee0dd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:15:20.345110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:20.345110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:20.345110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:20.345110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:20.345110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:20.345110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:20.345110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:20.345110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:20.345110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:20.345110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:20.345110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:20.345110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:15:20.345110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:15:20.345110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:15:20.346017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:15:20.346017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:20.346017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:20.346017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:20.346017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:20.346017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:20.346017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:20.346017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:20.346017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:20.346017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:20.346017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:20.346017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:20.346017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:20.346017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:20.346017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:20.346017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:20.346017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:20.347495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:15:20.347495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:15:20.347495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:15:20.347495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:15:20.347495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:15:20.347495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:15:20.347495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:15:20.347495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:20.347495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 530/1060] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:15:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:15:20.352554 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e59fd-c2e2-44a2-839e-d4500dee0dd3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:20.369536 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6c167-f906-47d9-aab9-1efef09d03b2 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:20.404616 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b311249f-0fad-4d53-b797-50a13b06de7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:20.405225 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311249f-0fad-4d53-b797-50a13b06de7d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:20.405508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311249f-0fad-4d53-b797-50a13b06de7d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:20.405783 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311249f-0fad-4d53-b797-50a13b06de7d None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:20.406409 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b311249f-0fad-4d53-b797-50a13b06de7d None admin] RBAC: Adding query filter params (type=image) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:20.406726 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b311249f-0fad-4d53-b797-50a13b06de7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:20.408106 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b311249f-0fad-4d53-b797-50a13b06de7d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:20.413716 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 531/1061] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:15:20 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:15:20.419180 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b311249f-0fad-4d53-b797-50a13b06de7d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:20.435803 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:20.458055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e59fd-c2e2-44a2-839e-d4500dee0dd3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:20.494030 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40096bc8-c141-40fb-a556-067312a5cb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:20.494640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40096bc8-c141-40fb-a556-067312a5cb79 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:20.494948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40096bc8-c141-40fb-a556-067312a5cb79 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:20.495227 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40096bc8-c141-40fb-a556-067312a5cb79 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:20.495919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40096bc8-c141-40fb-a556-067312a5cb79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:20.496225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40096bc8-c141-40fb-a556-067312a5cb79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:20.497636 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40096bc8-c141-40fb-a556-067312a5cb79 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:20.514466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 531/1062] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:15:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:15:20.515478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40096bc8-c141-40fb-a556-067312a5cb79 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:20.527400 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b311249f-0fad-4d53-b797-50a13b06de7d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:20.540853 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:20.564038 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9332f-9d64-4463-b54d-205b8f3735b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:20.564754 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9332f-9d64-4463-b54d-205b8f3735b0 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:20.565036 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9332f-9d64-4463-b54d-205b8f3735b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:20.565308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9332f-9d64-4463-b54d-205b8f3735b0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:20.565973 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f9332f-9d64-4463-b54d-205b8f3735b0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:20.567361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f9332f-9d64-4463-b54d-205b8f3735b0 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:20.617228 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40096bc8-c141-40fb-a556-067312a5cb79 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:20.650015 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf0853-468c-41c3-b07d-8887151cf078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:20.650675 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf0853-468c-41c3-b07d-8887151cf078 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:20.650984 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf0853-468c-41c3-b07d-8887151cf078 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:20.651260 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf0853-468c-41c3-b07d-8887151cf078 None admin] PATH_INFO: `/v3/projects/d18a5117a03b4f21ab3290a15680de66/users/1da355c8be2d496bb08feb0fe74e04b9/roles/dd2879946b3d4351acbaf8698b7966e8` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:20.653407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf0853-468c-41c3-b07d-8887151cf078 None admin] RBAC: Authorizing `identity:create_grant(project_id=d18a5117a03b4f21ab3290a15680de66, user_id=1da355c8be2d496bb08feb0fe74e04b9, role_id=dd2879946b3d4351acbaf8698b7966e8)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:20.654898 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf0853-468c-41c3-b07d-8887151cf078 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:20.662976 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 532/1063] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:15:20 2022] POST /identity/v3/endpoints => generated 327 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:15:20.731071 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 532/1064] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Sep 2 09:15:20 2022] PUT /identity/v3/projects/d18a5117a03b4f21ab3290a15680de66/users/1da355c8be2d496bb08feb0fe74e04b9/roles/dd2879946b3d4351acbaf8698b7966e8 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:15:21.027463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc3902-373f-4e69-9779-d6aac7ce20d7 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:21.027814 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc3902-373f-4e69-9779-d6aac7ce20d7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:21.028035 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc3902-373f-4e69-9779-d6aac7ce20d7 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:21.029486 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 533/1065] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:21.035360 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbc4e2-5ae8-441a-8b83-7ad239510683 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:21.035659 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbc4e2-5ae8-441a-8b83-7ad239510683 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:21.035941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbc4e2-5ae8-441a-8b83-7ad239510683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:21.053950 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1ecbc4e2-5ae8-441a-8b83-7ad239510683 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:21.055173 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1ecbc4e2-5ae8-441a-8b83-7ad239510683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:21.134153 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 533/1066] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:15:21.396181 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc3902-373f-4e69-9779-d6aac7ce20d7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:21.467982 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:21.499547 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc3902-373f-4e69-9779-d6aac7ce20d7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:21.533356 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e735b-b0e2-42a4-a738-f248dca142b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:21.533993 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e735b-b0e2-42a4-a738-f248dca142b0 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:21.534330 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e735b-b0e2-42a4-a738-f248dca142b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:21.534631 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e735b-b0e2-42a4-a738-f248dca142b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:21.545794 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e735b-b0e2-42a4-a738-f248dca142b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:15:21.551177 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-071e735b-b0e2-42a4-a738-f248dca142b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:15:21.551177 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:21.551177 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:21.551177 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:21.551177 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:21.551177 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:21.551177 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:21.551177 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:21.551177 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:21.551177 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:21.551177 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:21.551177 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:21.551177 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:15:21.551177 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:15:21.551177 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:15:21.551177 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:15:21.552076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:15:21.552076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:15:21.552076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:15:21.552076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:15:21.552076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:21.552076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:21.552076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:21.552076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:21.552076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:21.552076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:21.552076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:21.552076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:21.552076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:21.552076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:21.552076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:21.552076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:21.552954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:21.552954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:21.552954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:21.552954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:21.552954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:15:21.552954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:15:21.552954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:15:21.552954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:15:21.552954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:15:21.552954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:15:21.552954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:15:21.552954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:21.554777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 534/1067] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:15:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:21.558307 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbc4e2-5ae8-441a-8b83-7ad239510683 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:21.569472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:21.658007 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbc4e2-5ae8-441a-8b83-7ad239510683 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:21.697000 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf3e73-0414-4a20-a5e0-b67a0c132945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:21.697613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf3e73-0414-4a20-a5e0-b67a0c132945 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:21.697894 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf3e73-0414-4a20-a5e0-b67a0c132945 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:21.698164 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf3e73-0414-4a20-a5e0-b67a0c132945 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:21.698847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf3e73-0414-4a20-a5e0-b67a0c132945 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:21.699152 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf3e73-0414-4a20-a5e0-b67a0c132945 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:21.700575 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cf3e73-0414-4a20-a5e0-b67a0c132945 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:21.713694 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 534/1068] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:15:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:21.714688 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e735b-b0e2-42a4-a738-f248dca142b0 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:21.818050 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e735b-b0e2-42a4-a738-f248dca142b0 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:21.852877 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:21.858818 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b6e703-792a-422d-9abb-b2c139b85f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:21.859454 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b6e703-792a-422d-9abb-b2c139b85f0d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:21.866341 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b6e703-792a-422d-9abb-b2c139b85f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:21.866341 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b6e703-792a-422d-9abb-b2c139b85f0d None admin] PATH_INFO: `/v3/users/7213205233ed4e788addf4911af3fc8e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:21.876056 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b6e703-792a-422d-9abb-b2c139b85f0d None admin] RBAC: Authorizing `identity:get_user(user_id=7213205233ed4e788addf4911af3fc8e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:21.877481 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b6e703-792a-422d-9abb-b2c139b85f0d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:21.879501 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 535/1069] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:15:21 2022] GET /identity/v3/users/7213205233ed4e788addf4911af3fc8e => generated 285 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:21.885403 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf3e73-0414-4a20-a5e0-b67a0c132945 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:21.957643 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:21.987693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf3e73-0414-4a20-a5e0-b67a0c132945 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:22.021598 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de56d9-b50f-4436-9532-7200fef173a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:22.022210 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de56d9-b50f-4436-9532-7200fef173a8 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:22.022491 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de56d9-b50f-4436-9532-7200fef173a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:22.022791 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de56d9-b50f-4436-9532-7200fef173a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:22.089509 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de56d9-b50f-4436-9532-7200fef173a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:22.091035 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de56d9-b50f-4436-9532-7200fef173a8 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:22.158092 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:22.191161 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 535/1070] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:15:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 09:15:22.196267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b6e703-792a-422d-9abb-b2c139b85f0d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:22.296412 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b6e703-792a-422d-9abb-b2c139b85f0d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:22.331482 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83193b3f-2e64-41b8-865f-e708cf022f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:22.332372 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83193b3f-2e64-41b8-865f-e708cf022f2f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:22.332831 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83193b3f-2e64-41b8-865f-e708cf022f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:22.333309 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83193b3f-2e64-41b8-865f-e708cf022f2f None admin] PATH_INFO: `/v3/projects/60b1fc1dd5114cf3bf82b38cb201320f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:22.334592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83193b3f-2e64-41b8-865f-e708cf022f2f None admin] RBAC: Authorizing `identity:get_project(project_id=60b1fc1dd5114cf3bf82b38cb201320f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:22.336233 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83193b3f-2e64-41b8-865f-e708cf022f2f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:22.342981 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 536/1071] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:15:22 2022] GET /identity/v3/projects/60b1fc1dd5114cf3bf82b38cb201320f => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:22.344218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de56d9-b50f-4436-9532-7200fef173a8 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:22.443557 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de56d9-b50f-4436-9532-7200fef173a8 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:22.478145 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9579552-8b74-4aed-a271-032bf56d4488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:22.479076 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9579552-8b74-4aed-a271-032bf56d4488 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:22.479538 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9579552-8b74-4aed-a271-032bf56d4488 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:22.479987 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9579552-8b74-4aed-a271-032bf56d4488 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:22.480904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9579552-8b74-4aed-a271-032bf56d4488 None admin] RBAC: Adding query filter params (role.id=f886b8e8889549e896a8fb3426cce9e1, scope.project.id=60b1fc1dd5114cf3bf82b38cb201320f, user.id=7213205233ed4e788addf4911af3fc8e) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:22.481390 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9579552-8b74-4aed-a271-032bf56d4488 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:22.482996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9579552-8b74-4aed-a271-032bf56d4488 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:22.494250 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 536/1072] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:15:22 2022] GET /identity/v3/role_assignments?user.id=7213205233ed4e788addf4911af3fc8e&scope.project.id=60b1fc1dd5114cf3bf82b38cb201320f&role.id=f886b8e8889549e896a8fb3426cce9e1 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:15:22.607549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5ab64-36a2-4db1-83f4-473e668ecff1 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:22.608107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5ab64-36a2-4db1-83f4-473e668ecff1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:22.608107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba5ab64-36a2-4db1-83f4-473e668ecff1 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:22.609680 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 537/1073] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:22.615913 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b1c2c-21b4-443f-93f2-ac185c3ebdbd None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:22.616389 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b1c2c-21b4-443f-93f2-ac185c3ebdbd None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:22.616489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b1c2c-21b4-443f-93f2-ac185c3ebdbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:22.635785 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-955b1c2c-21b4-443f-93f2-ac185c3ebdbd None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:22.637058 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-955b1c2c-21b4-443f-93f2-ac185c3ebdbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:22.665822 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:22.669399 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38107de-03f3-421e-ada9-71f10bd948fd None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:22.669687 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38107de-03f3-421e-ada9-71f10bd948fd None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:22.669968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38107de-03f3-421e-ada9-71f10bd948fd None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:22.671338 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 538/1074] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:22.676612 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558a98e-79e3-4dbc-930a-d05f3eb6622a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:22.676895 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558a98e-79e3-4dbc-930a-d05f3eb6622a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:22.677171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558a98e-79e3-4dbc-930a-d05f3eb6622a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:22.695486 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a558a98e-79e3-4dbc-930a-d05f3eb6622a None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:22.696674 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a558a98e-79e3-4dbc-930a-d05f3eb6622a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:22.707482 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 537/1075] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:15:22.756441 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:22.772670 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 539/1076] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 09:15:22.977168 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b1c2c-21b4-443f-93f2-ac185c3ebdbd None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:23.045742 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558a98e-79e3-4dbc-930a-d05f3eb6622a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:23.082908 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b1c2c-21b4-443f-93f2-ac185c3ebdbd None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:23.119289 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc413588-4c32-448f-8198-a0ff2f471a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:23.119895 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc413588-4c32-448f-8198-a0ff2f471a49 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:23.120181 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc413588-4c32-448f-8198-a0ff2f471a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:23.120450 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc413588-4c32-448f-8198-a0ff2f471a49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:23.150312 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:23.159069 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558a98e-79e3-4dbc-930a-d05f3eb6622a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:23.186401 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc413588-4c32-448f-8198-a0ff2f471a49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:23.188459 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc413588-4c32-448f-8198-a0ff2f471a49 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:23.195608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd664d-9004-4619-b593-fe1b4984067a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:23.196249 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd664d-9004-4619-b593-fe1b4984067a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:23.196538 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd664d-9004-4619-b593-fe1b4984067a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:23.196815 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd664d-9004-4619-b593-fe1b4984067a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:23.207762 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd664d-9004-4619-b593-fe1b4984067a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:15:23.212550 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-85dd664d-9004-4619-b593-fe1b4984067a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 09:15:23.212550 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:23.212550 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:23.212550 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:23.212550 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:23.212550 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:23.212550 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:23.212550 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:23.212550 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:23.212550 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:23.212550 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:23.212550 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:23.212550 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:15:23.212550 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:15:23.212550 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:15:23.212550 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:15:23.213570 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:15:23.213570 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:15:23.213570 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:15:23.213570 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:15:23.213570 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:23.213570 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:23.213570 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:23.213570 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:23.213570 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:23.213570 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:23.213570 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:23.213570 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:23.213570 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:23.213570 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:23.213570 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:23.213570 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:23.214443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:23.214443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:23.214443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:23.214443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:23.214443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:15:23.214443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:15:23.214443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:15:23.214443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:15:23.214443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:15:23.214443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:15:23.214443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 02 09:15:23.214443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:23.214443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 540/1077] 10.222.0.57 () {56 vars in 1120 bytes} [Fri Sep 2 09:15:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:15:23.219346 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd664d-9004-4619-b593-fe1b4984067a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:23.269726 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:23.299236 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 538/1078] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:15:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 09:15:23.304902 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc413588-4c32-448f-8198-a0ff2f471a49 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:23.329834 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd664d-9004-4619-b593-fe1b4984067a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:23.367288 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:23.375287 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e15813-aa52-4301-a59c-31fb9307a563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:23.375902 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e15813-aa52-4301-a59c-31fb9307a563 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:23.376180 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e15813-aa52-4301-a59c-31fb9307a563 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:23.376453 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e15813-aa52-4301-a59c-31fb9307a563 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:23.377118 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e15813-aa52-4301-a59c-31fb9307a563 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:23.377420 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e15813-aa52-4301-a59c-31fb9307a563 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:23.379420 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e15813-aa52-4301-a59c-31fb9307a563 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:23.392247 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 541/1079] 10.222.0.57 () {56 vars in 1115 bytes} [Fri Sep 2 09:15:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:15:23.393257 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e15813-aa52-4301-a59c-31fb9307a563 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:23.409949 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc413588-4c32-448f-8198-a0ff2f471a49 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:23.451078 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a545063-25b2-4d78-919d-2acea712a5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:23.451706 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a545063-25b2-4d78-919d-2acea712a5d4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:23.451976 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a545063-25b2-4d78-919d-2acea712a5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:23.452242 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a545063-25b2-4d78-919d-2acea712a5d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:23.462951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a545063-25b2-4d78-919d-2acea712a5d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:23.464389 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a545063-25b2-4d78-919d-2acea712a5d4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:23.480854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-1a545063-25b2-4d78-919d-2acea712a5d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:15:23.480854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:23.480854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:23.480854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:23.480854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:23.480854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:23.480854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:23.480854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:23.480854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:23.480854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:23.480854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:23.480854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:23.480854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:15:23.480854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:15:23.480854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:15:23.480854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:15:23.481867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:23.481867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:23.481867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:23.481867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:23.481867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:23.481867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:23.481867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:23.481867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:23.481867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:23.481867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:23.481867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:23.481867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:23.481867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:23.481867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:23.481867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:23.481867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:23.481867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:15:23.482828 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:15:23.482828 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:15:23.482828 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:23.482828 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 539/1080] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:15:23.487764 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a545063-25b2-4d78-919d-2acea712a5d4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:23.504661 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e15813-aa52-4301-a59c-31fb9307a563 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:23.514440 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:23.539905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d19a50-1178-4940-a876-0cf5d6490fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:23.540526 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d19a50-1178-4940-a876-0cf5d6490fc5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:23.540805 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d19a50-1178-4940-a876-0cf5d6490fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:23.541104 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d19a50-1178-4940-a876-0cf5d6490fc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:23.552671 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d19a50-1178-4940-a876-0cf5d6490fc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:23.554427 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d19a50-1178-4940-a876-0cf5d6490fc5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:23.571421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-a1d19a50-1178-4940-a876-0cf5d6490fc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:15:23.571421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:23.571421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:23.571421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:23.571421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:23.571421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:23.571421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:23.571421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:23.571421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:23.571421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:23.571421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:23.571421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:23.571421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:15:23.571421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:15:23.571421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:15:23.571421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:15:23.573285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:23.573285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:23.573285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:23.573285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:23.573285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:23.573285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:23.573285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:23.573285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:23.573285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:23.573285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:23.573285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:23.573285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:23.573285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:23.573285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:23.573285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:23.573285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:23.573285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:15:23.574209 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:15:23.574209 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:15:23.574209 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:23.575332 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 542/1081] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:23.581010 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d19a50-1178-4940-a876-0cf5d6490fc5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:23.595225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a545063-25b2-4d78-919d-2acea712a5d4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:23.616513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:23.635433 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117139a6-69c8-4fa7-ac1a-7aceb7d82213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:23.636043 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117139a6-69c8-4fa7-ac1a-7aceb7d82213 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:23.636327 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117139a6-69c8-4fa7-ac1a-7aceb7d82213 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:23.636598 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117139a6-69c8-4fa7-ac1a-7aceb7d82213 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:23.637243 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117139a6-69c8-4fa7-ac1a-7aceb7d82213 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:23.637581 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117139a6-69c8-4fa7-ac1a-7aceb7d82213 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:23.638958 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117139a6-69c8-4fa7-ac1a-7aceb7d82213 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:23.651459 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 540/1082] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:15:23.689234 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d19a50-1178-4940-a876-0cf5d6490fc5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:23.726287 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0e293-51c9-4a8e-a693-5ad1ad22aa85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:23.726971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0e293-51c9-4a8e-a693-5ad1ad22aa85 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:23.727250 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0e293-51c9-4a8e-a693-5ad1ad22aa85 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:23.727517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0e293-51c9-4a8e-a693-5ad1ad22aa85 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:23.728174 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0e293-51c9-4a8e-a693-5ad1ad22aa85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:23.728476 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0e293-51c9-4a8e-a693-5ad1ad22aa85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:23.729834 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c0e293-51c9-4a8e-a693-5ad1ad22aa85 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:23.747056 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 543/1083] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:15:23.749030 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117139a6-69c8-4fa7-ac1a-7aceb7d82213 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:23.860971 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117139a6-69c8-4fa7-ac1a-7aceb7d82213 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:23.897173 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ef3d1-f51d-4bf0-a7b8-613126f01a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:23.897802 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ef3d1-f51d-4bf0-a7b8-613126f01a55 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:23.898079 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ef3d1-f51d-4bf0-a7b8-613126f01a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:23.898349 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ef3d1-f51d-4bf0-a7b8-613126f01a55 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:23.905600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ef3d1-f51d-4bf0-a7b8-613126f01a55 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:23.907050 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ef3d1-f51d-4bf0-a7b8-613126f01a55 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:23.918857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-4b0ef3d1-f51d-4bf0-a7b8-613126f01a55 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 02 09:15:23.918857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:23.918857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:23.918857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:23.918857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:23.918857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:23.918857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:23.918857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:23.918857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:23.918857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:23.918857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:23.918857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:23.918857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:15:23.918857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:15:23.918857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:15:23.918857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:15:23.919821 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:23.919821 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:23.919821 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:15:23.919821 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:15:23.919821 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:15:23.919821 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:15:23.919821 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:23.919821 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:23.919821 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:23.919821 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:23.919821 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:23.919821 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:23.919821 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:23.919821 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:23.919821 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:23.919821 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:23.919821 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:23.919821 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:23.920775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:23.920775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:23.920775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:15:23.920775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:15:23.920775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:15:23.920775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:15:23.920775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:15:23.920775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:15:23.920775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 02 09:15:23.920775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:23.920775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 541/1084] 10.222.0.57 () {56 vars in 1093 bytes} [Fri Sep 2 09:15:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:23.926607 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0e293-51c9-4a8e-a693-5ad1ad22aa85 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:23.951518 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:24.031321 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c0e293-51c9-4a8e-a693-5ad1ad22aa85 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:24.064612 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a7ccac-20ce-4e24-ba28-411960ca2441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:24.065217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7ccac-20ce-4e24-ba28-411960ca2441 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:24.065589 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7ccac-20ce-4e24-ba28-411960ca2441 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:24.065860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7ccac-20ce-4e24-ba28-411960ca2441 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:24.066581 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a7ccac-20ce-4e24-ba28-411960ca2441 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:24.066914 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a7ccac-20ce-4e24-ba28-411960ca2441 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:24.068323 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a7ccac-20ce-4e24-ba28-411960ca2441 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:24.085762 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 544/1085] 10.222.0.57 () {56 vars in 1133 bytes} [Fri Sep 2 09:15:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:24.091283 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ef3d1-f51d-4bf0-a7b8-613126f01a55 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:24.125610 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:24.188137 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ef3d1-f51d-4bf0-a7b8-613126f01a55 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:24.221175 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cffafe-ca3b-4c3b-865e-65f1b9a06e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:24.221803 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cffafe-ca3b-4c3b-865e-65f1b9a06e64 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:24.222083 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cffafe-ca3b-4c3b-865e-65f1b9a06e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:24.222354 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cffafe-ca3b-4c3b-865e-65f1b9a06e64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:24.287129 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cffafe-ca3b-4c3b-865e-65f1b9a06e64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:24.288570 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cffafe-ca3b-4c3b-865e-65f1b9a06e64 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:24.321107 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:24.371258 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38306af-3af1-4fef-b1f4-cfcb2140a922 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:24.371539 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38306af-3af1-4fef-b1f4-cfcb2140a922 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:24.371819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38306af-3af1-4fef-b1f4-cfcb2140a922 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:24.373158 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 545/1086] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:15:24.378606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae7278-585c-474e-b024-9e7670ecb854 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:24.378877 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae7278-585c-474e-b024-9e7670ecb854 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:24.379156 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae7278-585c-474e-b024-9e7670ecb854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:24.390547 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 542/1087] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:15:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 09:15:24.391692 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cffafe-ca3b-4c3b-865e-65f1b9a06e64 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:24.397311 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-58ae7278-585c-474e-b024-9e7670ecb854 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:24.398505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-58ae7278-585c-474e-b024-9e7670ecb854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:24.465461 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 546/1088] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 09:15:24.489544 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cffafe-ca3b-4c3b-865e-65f1b9a06e64 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:24.523702 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b9c49-cbc7-4ca4-a4d9-81ac74726fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:24.524332 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b9c49-cbc7-4ca4-a4d9-81ac74726fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:24.524614 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b9c49-cbc7-4ca4-a4d9-81ac74726fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:24.524890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b9c49-cbc7-4ca4-a4d9-81ac74726fb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:24.527824 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:24.535435 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b9c49-cbc7-4ca4-a4d9-81ac74726fb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:24.536864 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b9c49-cbc7-4ca4-a4d9-81ac74726fb2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:24.551920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-086b9c49-cbc7-4ca4-a4d9-81ac74726fb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:15:24.551920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:24.551920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:24.551920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:24.551920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:24.551920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:24.551920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:24.551920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:24.551920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:24.551920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:24.551920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:24.551920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:24.551920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:15:24.551920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:15:24.551920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:15:24.551920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:15:24.553045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:24.553045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:24.553045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:24.553045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:24.553045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:24.553045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:24.553045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:24.553045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:24.553045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:24.553045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:24.553045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:24.553045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:24.553045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:24.553045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:24.553045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:24.553045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:24.553045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:15:24.553934 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:15:24.553934 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:15:24.553934 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:24.553934 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 543/1089] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:15:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:24.558839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae7278-585c-474e-b024-9e7670ecb854 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:24.570241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:24.657765 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae7278-585c-474e-b024-9e7670ecb854 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:24.691101 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d979f0-2159-4640-9dca-6684306c79f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:24.691700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d979f0-2159-4640-9dca-6684306c79f1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:24.691973 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d979f0-2159-4640-9dca-6684306c79f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:24.692244 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d979f0-2159-4640-9dca-6684306c79f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:24.692880 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d979f0-2159-4640-9dca-6684306c79f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:24.693181 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d979f0-2159-4640-9dca-6684306c79f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:24.694528 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d979f0-2159-4640-9dca-6684306c79f1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:24.705769 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 547/1090] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:24.711639 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b9c49-cbc7-4ca4-a4d9-81ac74726fb2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:24.730083 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d979f0-2159-4640-9dca-6684306c79f1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:24.795194 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:24.811231 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b9c49-cbc7-4ca4-a4d9-81ac74726fb2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:24.828971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d979f0-2159-4640-9dca-6684306c79f1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:24.846220 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9454b1-d451-44f4-bd29-20111b02a972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:24.847405 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9454b1-d451-44f4-bd29-20111b02a972 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:24.847685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9454b1-d451-44f4-bd29-20111b02a972 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:24.847959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9454b1-d451-44f4-bd29-20111b02a972 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:24.854750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9454b1-d451-44f4-bd29-20111b02a972 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:24.856269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9454b1-d451-44f4-bd29-20111b02a972 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:24.863874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2113dc4-56af-44ac-a489-a1c0dcc030c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:24.864501 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2113dc4-56af-44ac-a489-a1c0dcc030c8 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:24.864784 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2113dc4-56af-44ac-a489-a1c0dcc030c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:24.865062 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2113dc4-56af-44ac-a489-a1c0dcc030c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:24.867357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-bf9454b1-d451-44f4-bd29-20111b02a972 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:15:24.867357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:24.867357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:24.867357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:24.867357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:24.867357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:24.867357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:24.867357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:24.867357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:24.867357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:24.867357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:24.867357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:24.867357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:15:24.867357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:15:24.867357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:15:24.868218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:15:24.868218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:24.868218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:24.868218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:24.868218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:24.868218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:24.868218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:24.868218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:24.868218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:24.868218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:24.868218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:24.868218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:24.868218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:24.868218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:24.868218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:24.868218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:24.868218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:24.869119 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:15:24.869119 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:15:24.869119 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:15:24.869119 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:15:24.869119 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:15:24.869119 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:15:24.869119 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:15:24.869119 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:24.869119 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 544/1091] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:15:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:24.874134 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9454b1-d451-44f4-bd29-20111b02a972 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:24.878774 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2113dc4-56af-44ac-a489-a1c0dcc030c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:15:24.884214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-f2113dc4-56af-44ac-a489-a1c0dcc030c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:15:24.884214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:24.884214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:24.884214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:24.884214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:24.884214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:24.884214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:24.884214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:24.884214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:24.884214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:24.884214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:24.884214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:24.884214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:15:24.884214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:15:24.884214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:15:24.884214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:15:24.885128 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:15:24.885128 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:15:24.885128 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:15:24.885128 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:15:24.885128 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:24.885128 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:24.885128 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:24.885128 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:24.885128 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:24.885128 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:24.885128 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:24.885128 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:24.885128 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:24.885128 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:24.885128 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:24.885128 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:24.886017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:24.886017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:24.886017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:24.886017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:24.886017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:15:24.886017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:15:24.886017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:15:24.886017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:15:24.886017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:15:24.886017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:15:24.886017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:15:24.886017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:24.886017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 548/1092] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:15:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:15:24.891144 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2113dc4-56af-44ac-a489-a1c0dcc030c8 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:24.929745 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:24.983279 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9454b1-d451-44f4-bd29-20111b02a972 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:24.997353 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2113dc4-56af-44ac-a489-a1c0dcc030c8 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:25.016959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae63978-0ff0-4643-bc2a-4ccfdf5cd992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:25.017566 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae63978-0ff0-4643-bc2a-4ccfdf5cd992 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:25.017843 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae63978-0ff0-4643-bc2a-4ccfdf5cd992 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:25.018110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae63978-0ff0-4643-bc2a-4ccfdf5cd992 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:25.019461 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae63978-0ff0-4643-bc2a-4ccfdf5cd992 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:25.019766 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae63978-0ff0-4643-bc2a-4ccfdf5cd992 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:25.021181 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae63978-0ff0-4643-bc2a-4ccfdf5cd992 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:25.024680 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:25.032355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1b7b5-d07a-4ba0-bfda-093231174822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:25.032968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1b7b5-d07a-4ba0-bfda-093231174822 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:25.033082 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 545/1093] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:15:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:15:25.033323 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1b7b5-d07a-4ba0-bfda-093231174822 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:25.033619 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1b7b5-d07a-4ba0-bfda-093231174822 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:25.034359 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b1b7b5-d07a-4ba0-bfda-093231174822 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:25.034673 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b1b7b5-d07a-4ba0-bfda-093231174822 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:25.043034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b1b7b5-d07a-4ba0-bfda-093231174822 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:25.043191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae63978-0ff0-4643-bc2a-4ccfdf5cd992 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:25.049468 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 549/1094] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:15:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:15:25.050486 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1b7b5-d07a-4ba0-bfda-093231174822 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:25.137610 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae63978-0ff0-4643-bc2a-4ccfdf5cd992 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:25.147213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:25.149459 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1b7b5-d07a-4ba0-bfda-093231174822 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:25.171131 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2953edc-6a1a-43cc-81ff-42c4747c0576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:25.171736 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2953edc-6a1a-43cc-81ff-42c4747c0576 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:25.172022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2953edc-6a1a-43cc-81ff-42c4747c0576 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:25.172296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2953edc-6a1a-43cc-81ff-42c4747c0576 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:25.173019 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2953edc-6a1a-43cc-81ff-42c4747c0576 None admin] RBAC: Adding query filter params (role.id=dd2879946b3d4351acbaf8698b7966e8, scope.project.id=d18a5117a03b4f21ab3290a15680de66, user.id=1da355c8be2d496bb08feb0fe74e04b9) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:25.173331 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2953edc-6a1a-43cc-81ff-42c4747c0576 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:25.174808 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2953edc-6a1a-43cc-81ff-42c4747c0576 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:25.181575 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 546/1095] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:15:25 2022] GET /identity/v3/role_assignments?user.id=1da355c8be2d496bb08feb0fe74e04b9&scope.project.id=d18a5117a03b4f21ab3290a15680de66&role.id=dd2879946b3d4351acbaf8698b7966e8 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:15:25.182531 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174712c-ba97-4201-837d-982c8b25dc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:25.183205 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174712c-ba97-4201-837d-982c8b25dc4f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:25.183489 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174712c-ba97-4201-837d-982c8b25dc4f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:25.183762 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174712c-ba97-4201-837d-982c8b25dc4f None admin] PATH_INFO: `/v3/users/7213205233ed4e788addf4911af3fc8e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:25.184971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b174712c-ba97-4201-837d-982c8b25dc4f None admin] RBAC: Authorizing `identity:get_user(user_id=7213205233ed4e788addf4911af3fc8e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:25.186416 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b174712c-ba97-4201-837d-982c8b25dc4f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:25.188113 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 550/1096] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:15:25 2022] GET /identity/v3/users/7213205233ed4e788addf4911af3fc8e => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:25.194478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2953edc-6a1a-43cc-81ff-42c4747c0576 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:25.303379 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2953edc-6a1a-43cc-81ff-42c4747c0576 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:25.338995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b5535-9a76-4dd4-8ab7-40ad6378be5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:25.339627 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b5535-9a76-4dd4-8ab7-40ad6378be5f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:25.339908 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b5535-9a76-4dd4-8ab7-40ad6378be5f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:25.340177 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b5535-9a76-4dd4-8ab7-40ad6378be5f None admin] PATH_INFO: `/v3/projects/60b1fc1dd5114cf3bf82b38cb201320f` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:25.341297 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b5535-9a76-4dd4-8ab7-40ad6378be5f None admin] RBAC: Authorizing `identity:get_project(project_id=60b1fc1dd5114cf3bf82b38cb201320f)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:25.342730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b5535-9a76-4dd4-8ab7-40ad6378be5f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:25.344358 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 547/1097] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:15:25 2022] GET /identity/v3/projects/60b1fc1dd5114cf3bf82b38cb201320f => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:25.350573 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174712c-ba97-4201-837d-982c8b25dc4f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:25.411357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:25.453132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174712c-ba97-4201-837d-982c8b25dc4f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:25.486903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483b864-e649-4149-8e62-210f90aafebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:25.488052 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483b864-e649-4149-8e62-210f90aafebf None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:25.488330 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483b864-e649-4149-8e62-210f90aafebf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:25.488600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483b864-e649-4149-8e62-210f90aafebf None admin] PATH_INFO: `/v3/projects/60b1fc1dd5114cf3bf82b38cb201320f/users/7213205233ed4e788addf4911af3fc8e/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:25.490463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483b864-e649-4149-8e62-210f90aafebf None admin] RBAC: Authorizing `identity:create_grant(project_id=60b1fc1dd5114cf3bf82b38cb201320f, user_id=7213205233ed4e788addf4911af3fc8e, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:25.491978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483b864-e649-4149-8e62-210f90aafebf None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:25.570929 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 551/1098] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Sep 2 09:15:25 2022] PUT /identity/v3/projects/60b1fc1dd5114cf3bf82b38cb201320f/users/7213205233ed4e788addf4911af3fc8e/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:27.450125 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25d086-8d90-4603-85f3-0d95f8156760 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:27.450669 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25d086-8d90-4603-85f3-0d95f8156760 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:27.450669 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25d086-8d90-4603-85f3-0d95f8156760 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:27.452018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 548/1099] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:27.457873 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa5d74-b6db-4c23-85aa-f25e156b40b7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:27.458270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa5d74-b6db-4c23-85aa-f25e156b40b7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:27.458410 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fa5d74-b6db-4c23-85aa-f25e156b40b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:27.476874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-08fa5d74-b6db-4c23-85aa-f25e156b40b7 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:27.478042 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-08fa5d74-b6db-4c23-85aa-f25e156b40b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:27.541327 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 552/1100] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 09:15:27.803709 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25d086-8d90-4603-85f3-0d95f8156760 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:27.845628 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:27.902208 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25d086-8d90-4603-85f3-0d95f8156760 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:27.933960 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dfe7c7-20e4-463e-990c-b3c3176125c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:27.934574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dfe7c7-20e4-463e-990c-b3c3176125c2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:27.934881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dfe7c7-20e4-463e-990c-b3c3176125c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:27.935149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dfe7c7-20e4-463e-990c-b3c3176125c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:27.945551 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dfe7c7-20e4-463e-990c-b3c3176125c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:15:27.950221 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-65dfe7c7-20e4-463e-990c-b3c3176125c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:15:27.950221 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:27.950221 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:27.950221 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:27.950221 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:27.950221 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:27.950221 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:27.950221 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:27.950221 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:27.950221 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:27.950221 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:27.950221 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:27.950221 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:15:27.950221 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:15:27.950221 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:15:27.950221 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:15:27.951366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:15:27.951366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:15:27.951366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:15:27.951366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:15:27.951366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:27.951366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:27.951366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:27.951366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:27.951366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:27.951366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:27.951366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:27.951366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:27.951366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:27.951366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:27.951366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:27.951366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:27.952224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:27.952224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:27.952224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:27.952224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:27.952224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:15:27.952224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:15:27.952224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:15:27.952224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:15:27.952224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:15:27.952224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:15:27.952224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:15:27.952224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:27.952224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 549/1101] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:15:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:27.957893 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa5d74-b6db-4c23-85aa-f25e156b40b7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:28.004115 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:28.056392 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fa5d74-b6db-4c23-85aa-f25e156b40b7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:28.090406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6125f8f-84ef-4105-8205-7255a2531ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:28.091036 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6125f8f-84ef-4105-8205-7255a2531ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:28.091318 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6125f8f-84ef-4105-8205-7255a2531ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:28.091588 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6125f8f-84ef-4105-8205-7255a2531ea4 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:28.092242 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6125f8f-84ef-4105-8205-7255a2531ea4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:28.092542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6125f8f-84ef-4105-8205-7255a2531ea4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:28.093888 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6125f8f-84ef-4105-8205-7255a2531ea4 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:28.101680 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 553/1102] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:15:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:28.107465 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dfe7c7-20e4-463e-990c-b3c3176125c2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:28.200784 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:28.203566 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dfe7c7-20e4-463e-990c-b3c3176125c2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:28.235430 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a1e36-034c-4bf7-bbff-e130083be131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:28.236039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a1e36-034c-4bf7-bbff-e130083be131 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:28.236315 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a1e36-034c-4bf7-bbff-e130083be131 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:28.236616 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a1e36-034c-4bf7-bbff-e130083be131 None admin] PATH_INFO: `/v3/users/7213205233ed4e788addf4911af3fc8e` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:28.238351 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a1e36-034c-4bf7-bbff-e130083be131 None admin] RBAC: Authorizing `identity:get_user(user_id=7213205233ed4e788addf4911af3fc8e)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:28.240009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a1e36-034c-4bf7-bbff-e130083be131 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:28.241880 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 550/1103] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:15:28 2022] GET /identity/v3/users/7213205233ed4e788addf4911af3fc8e => generated 285 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:28.247277 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6125f8f-84ef-4105-8205-7255a2531ea4 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:28.344707 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6125f8f-84ef-4105-8205-7255a2531ea4 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:28.355147 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:28.377450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bdeca-4334-4d1a-a9cb-1b496584707f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:28.378069 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bdeca-4334-4d1a-a9cb-1b496584707f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:28.378351 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bdeca-4334-4d1a-a9cb-1b496584707f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:28.378625 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bdeca-4334-4d1a-a9cb-1b496584707f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:28.443387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940bdeca-4334-4d1a-a9cb-1b496584707f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:28.445082 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940bdeca-4334-4d1a-a9cb-1b496584707f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:28.539606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 554/1104] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:15:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 09:15:28.545399 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a1e36-034c-4bf7-bbff-e130083be131 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:28.652340 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a1e36-034c-4bf7-bbff-e130083be131 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:28.683817 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f27fa78-cbe3-4378-82f2-ec0056202891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:28.684434 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27fa78-cbe3-4378-82f2-ec0056202891 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:28.684713 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27fa78-cbe3-4378-82f2-ec0056202891 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:28.684981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27fa78-cbe3-4378-82f2-ec0056202891 None admin] PATH_INFO: `/v3/projects/60b1fc1dd5114cf3bf82b38cb201320f` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:28.686063 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f27fa78-cbe3-4378-82f2-ec0056202891 None admin] RBAC: Authorizing `identity:get_project(project_id=60b1fc1dd5114cf3bf82b38cb201320f)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:28.687487 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f27fa78-cbe3-4378-82f2-ec0056202891 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:28.689069 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 551/1105] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:15:28 2022] GET /identity/v3/projects/60b1fc1dd5114cf3bf82b38cb201320f => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:28.695712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bdeca-4334-4d1a-a9cb-1b496584707f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:28.707551 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:28.792205 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bdeca-4334-4d1a-a9cb-1b496584707f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:28.824719 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc2173-d952-46a4-8725-e1356a761676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:28.825306 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc2173-d952-46a4-8725-e1356a761676 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:28.825582 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc2173-d952-46a4-8725-e1356a761676 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:28.825852 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc2173-d952-46a4-8725-e1356a761676 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:28.826558 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dc2173-d952-46a4-8725-e1356a761676 None admin] RBAC: Adding query filter params (role.id=f886b8e8889549e896a8fb3426cce9e1, scope.project.id=60b1fc1dd5114cf3bf82b38cb201320f, user.id=7213205233ed4e788addf4911af3fc8e) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:28.826891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dc2173-d952-46a4-8725-e1356a761676 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:28.828293 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dc2173-d952-46a4-8725-e1356a761676 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:28.834375 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 555/1106] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:15:28 2022] GET /identity/v3/role_assignments?user.id=7213205233ed4e788addf4911af3fc8e&scope.project.id=60b1fc1dd5114cf3bf82b38cb201320f&role.id=f886b8e8889549e896a8fb3426cce9e1 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:15:30.741635 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1e29c-5206-41dd-9369-e34bf3cc132d None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:30.742163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1e29c-5206-41dd-9369-e34bf3cc132d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:30.742163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b1e29c-5206-41dd-9369-e34bf3cc132d None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:30.743546 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 552/1107] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:30.748634 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde3a87-1ce7-46fb-8c42-f0bfbff43a23 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:30.749045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde3a87-1ce7-46fb-8c42-f0bfbff43a23 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:30.749175 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cde3a87-1ce7-46fb-8c42-f0bfbff43a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:30.767409 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7cde3a87-1ce7-46fb-8c42-f0bfbff43a23 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:30.768575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7cde3a87-1ce7-46fb-8c42-f0bfbff43a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:30.832769 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 556/1108] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:15:31.107243 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1e29c-5206-41dd-9369-e34bf3cc132d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:31.142421 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:31.205759 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b1e29c-5206-41dd-9369-e34bf3cc132d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:31.238679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a056f-d423-4d89-ab12-a9d1990a257d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:31.239316 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a056f-d423-4d89-ab12-a9d1990a257d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:31.239591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a056f-d423-4d89-ab12-a9d1990a257d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:31.239861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a056f-d423-4d89-ab12-a9d1990a257d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:31.240940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a056f-d423-4d89-ab12-a9d1990a257d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:31.242347 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a056f-d423-4d89-ab12-a9d1990a257d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:31.243900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 553/1109] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:15:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:31.249539 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde3a87-1ce7-46fb-8c42-f0bfbff43a23 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:31.252620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:31.344605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cde3a87-1ce7-46fb-8c42-f0bfbff43a23 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:31.377214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e06eb-b3d5-4d06-b2fb-43fe2e5f5156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:31.377777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e06eb-b3d5-4d06-b2fb-43fe2e5f5156 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:31.378054 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e06eb-b3d5-4d06-b2fb-43fe2e5f5156 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:31.378322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e06eb-b3d5-4d06-b2fb-43fe2e5f5156 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:31.378998 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e06eb-b3d5-4d06-b2fb-43fe2e5f5156 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:31.380401 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e06eb-b3d5-4d06-b2fb-43fe2e5f5156 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:31.470238 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 557/1110] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Sep 2 09:15:31 2022] POST /identity/v3/users => generated 286 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:15:33.388821 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc949ce-3671-43bd-861d-5177bad94531 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:33.389360 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc949ce-3671-43bd-861d-5177bad94531 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:33.389360 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc949ce-3671-43bd-861d-5177bad94531 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:33.390724 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 554/1111] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:33.396333 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f144f9-fc69-4795-b714-fdb78ba791ed None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:33.396715 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f144f9-fc69-4795-b714-fdb78ba791ed None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:33.396885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f144f9-fc69-4795-b714-fdb78ba791ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:33.414520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d1f144f9-fc69-4795-b714-fdb78ba791ed None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:33.415732 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d1f144f9-fc69-4795-b714-fdb78ba791ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:33.469479 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:33.480232 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 558/1112] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:15:33.746238 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc949ce-3671-43bd-861d-5177bad94531 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:33.843788 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc949ce-3671-43bd-861d-5177bad94531 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:33.846598 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:33.877625 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35753e-1dfd-439a-9964-95e990219227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:33.878368 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35753e-1dfd-439a-9964-95e990219227 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:33.878693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35753e-1dfd-439a-9964-95e990219227 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:33.879016 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35753e-1dfd-439a-9964-95e990219227 None admin] PATH_INFO: `/v3/roles/f2fe9fb31024471e9e2bea2b12715de6` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:33.880631 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb35753e-1dfd-439a-9964-95e990219227 None admin] RBAC: Authorizing `identity:get_role(role_id=f2fe9fb31024471e9e2bea2b12715de6)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:33.881975 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb35753e-1dfd-439a-9964-95e990219227 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:33.883298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 555/1113] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:15:33 2022] GET /identity/v3/roles/f2fe9fb31024471e9e2bea2b12715de6 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:33.889030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f144f9-fc69-4795-b714-fdb78ba791ed None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:33.984479 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f144f9-fc69-4795-b714-fdb78ba791ed None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:34.017249 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac575c53-dec4-436c-a6aa-7ca5c0627acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:34.017836 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac575c53-dec4-436c-a6aa-7ca5c0627acf None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:34.018119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac575c53-dec4-436c-a6aa-7ca5c0627acf None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:34.018396 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac575c53-dec4-436c-a6aa-7ca5c0627acf None admin] PATH_INFO: `/v3/users/0f69025973b24e1f8a869932ce5e235e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:34.030864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac575c53-dec4-436c-a6aa-7ca5c0627acf None admin] RBAC: Authorizing `identity:get_user(user_id=0f69025973b24e1f8a869932ce5e235e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:34.032296 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac575c53-dec4-436c-a6aa-7ca5c0627acf None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:34.034374 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 559/1114] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:15:33 2022] GET /identity/v3/users/0f69025973b24e1f8a869932ce5e235e => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:34.040061 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35753e-1dfd-439a-9964-95e990219227 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:34.136123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35753e-1dfd-439a-9964-95e990219227 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:34.168485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8310e947-0352-45e5-817f-3870a94d38f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:34.169068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8310e947-0352-45e5-817f-3870a94d38f7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:34.169345 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8310e947-0352-45e5-817f-3870a94d38f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:34.169612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8310e947-0352-45e5-817f-3870a94d38f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:34.208086 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:34.233461 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8310e947-0352-45e5-817f-3870a94d38f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:34.234921 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8310e947-0352-45e5-817f-3870a94d38f7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:34.329250 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 556/1115] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:15:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 09:15:34.335610 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac575c53-dec4-436c-a6aa-7ca5c0627acf None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:34.383360 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:34.433371 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac575c53-dec4-436c-a6aa-7ca5c0627acf None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:34.466747 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78eb2c8-bbbd-4424-832b-2b32f8543a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:34.467370 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78eb2c8-bbbd-4424-832b-2b32f8543a5f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:34.467650 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78eb2c8-bbbd-4424-832b-2b32f8543a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:34.467925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78eb2c8-bbbd-4424-832b-2b32f8543a5f None admin] PATH_INFO: `/v3/projects/60b1fc1dd5114cf3bf82b38cb201320f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:34.469025 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78eb2c8-bbbd-4424-832b-2b32f8543a5f None admin] RBAC: Authorizing `identity:get_project(project_id=60b1fc1dd5114cf3bf82b38cb201320f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:34.470434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78eb2c8-bbbd-4424-832b-2b32f8543a5f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:34.472091 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 560/1116] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:15:34 2022] GET /identity/v3/projects/60b1fc1dd5114cf3bf82b38cb201320f => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:34.478052 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8310e947-0352-45e5-817f-3870a94d38f7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:34.544268 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:34.573132 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8310e947-0352-45e5-817f-3870a94d38f7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:34.605930 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0f2fa-6b18-46e2-99cb-6e18fd7b9a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:34.606526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0f2fa-6b18-46e2-99cb-6e18fd7b9a1e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:34.606838 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0f2fa-6b18-46e2-99cb-6e18fd7b9a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:34.607106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0f2fa-6b18-46e2-99cb-6e18fd7b9a1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:34.607823 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0f2fa-6b18-46e2-99cb-6e18fd7b9a1e None admin] RBAC: Adding query filter params (role.id=f2fe9fb31024471e9e2bea2b12715de6, scope.project.id=60b1fc1dd5114cf3bf82b38cb201320f, user.id=0f69025973b24e1f8a869932ce5e235e) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:34.608132 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0f2fa-6b18-46e2-99cb-6e18fd7b9a1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:34.609566 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0f2fa-6b18-46e2-99cb-6e18fd7b9a1e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:34.615278 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 557/1117] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:15:34 2022] GET /identity/v3/role_assignments?user.id=0f69025973b24e1f8a869932ce5e235e&scope.project.id=60b1fc1dd5114cf3bf82b38cb201320f&role.id=f2fe9fb31024471e9e2bea2b12715de6 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:36.470455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afac22-84e5-4155-af16-0e5375066f59 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:36.471030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afac22-84e5-4155-af16-0e5375066f59 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:36.471030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afac22-84e5-4155-af16-0e5375066f59 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:36.472392 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 561/1118] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:36.477794 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a982aa-2292-45db-9f45-8a02a57d58e5 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:36.478160 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a982aa-2292-45db-9f45-8a02a57d58e5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:36.478343 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a982aa-2292-45db-9f45-8a02a57d58e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:36.496391 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c4a982aa-2292-45db-9f45-8a02a57d58e5 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:36.497560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c4a982aa-2292-45db-9f45-8a02a57d58e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:36.560476 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 558/1119] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:15:36.837831 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afac22-84e5-4155-af16-0e5375066f59 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:36.935825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afac22-84e5-4155-af16-0e5375066f59 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:36.951599 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:36.968165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8363c81-7327-46a4-9893-33a730934640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:36.969856 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8363c81-7327-46a4-9893-33a730934640 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:36.969856 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8363c81-7327-46a4-9893-33a730934640 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:36.969856 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8363c81-7327-46a4-9893-33a730934640 None admin] PATH_INFO: `/v3/roles/f2fe9fb31024471e9e2bea2b12715de6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:36.971227 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8363c81-7327-46a4-9893-33a730934640 None admin] RBAC: Authorizing `identity:get_role(role_id=f2fe9fb31024471e9e2bea2b12715de6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:36.972583 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8363c81-7327-46a4-9893-33a730934640 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:36.973878 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 562/1120] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:15:36 2022] GET /identity/v3/roles/f2fe9fb31024471e9e2bea2b12715de6 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:36.979694 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a982aa-2292-45db-9f45-8a02a57d58e5 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:37.041569 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:37.075021 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a982aa-2292-45db-9f45-8a02a57d58e5 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:37.107824 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60656191-829b-47bb-a4b9-bec085db8a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:37.108444 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60656191-829b-47bb-a4b9-bec085db8a62 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:37.108753 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60656191-829b-47bb-a4b9-bec085db8a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:37.109025 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60656191-829b-47bb-a4b9-bec085db8a62 None admin] PATH_INFO: `/v3/users/0f69025973b24e1f8a869932ce5e235e` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:37.110133 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60656191-829b-47bb-a4b9-bec085db8a62 None admin] RBAC: Authorizing `identity:get_user(user_id=0f69025973b24e1f8a869932ce5e235e)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:37.112155 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60656191-829b-47bb-a4b9-bec085db8a62 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:37.113701 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 559/1121] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:15:36 2022] GET /identity/v3/users/0f69025973b24e1f8a869932ce5e235e => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:37.119322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8363c81-7327-46a4-9893-33a730934640 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:37.214643 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8363c81-7327-46a4-9893-33a730934640 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:37.247386 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc886d1-32f0-4196-9cbe-a721f8e5497c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:37.247994 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc886d1-32f0-4196-9cbe-a721f8e5497c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:37.248270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc886d1-32f0-4196-9cbe-a721f8e5497c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:37.248537 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc886d1-32f0-4196-9cbe-a721f8e5497c None admin] PATH_INFO: `/v3/projects/60b1fc1dd5114cf3bf82b38cb201320f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:37.249624 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc886d1-32f0-4196-9cbe-a721f8e5497c None admin] RBAC: Authorizing `identity:get_project(project_id=60b1fc1dd5114cf3bf82b38cb201320f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:37.251042 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc886d1-32f0-4196-9cbe-a721f8e5497c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:37.252654 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 563/1122] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:15:37 2022] GET /identity/v3/projects/60b1fc1dd5114cf3bf82b38cb201320f => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:37.258431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60656191-829b-47bb-a4b9-bec085db8a62 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:37.355939 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60656191-829b-47bb-a4b9-bec085db8a62 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:37.388330 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071fa51-9137-40d8-b741-b66a9e9674c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:37.388916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071fa51-9137-40d8-b741-b66a9e9674c2 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:37.389198 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071fa51-9137-40d8-b741-b66a9e9674c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:37.389475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071fa51-9137-40d8-b741-b66a9e9674c2 None admin] PATH_INFO: `/v3/projects/60b1fc1dd5114cf3bf82b38cb201320f/users/0f69025973b24e1f8a869932ce5e235e/roles/f2fe9fb31024471e9e2bea2b12715de6` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:37.391624 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f071fa51-9137-40d8-b741-b66a9e9674c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=60b1fc1dd5114cf3bf82b38cb201320f, user_id=0f69025973b24e1f8a869932ce5e235e, role_id=f2fe9fb31024471e9e2bea2b12715de6)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:37.393569 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f071fa51-9137-40d8-b741-b66a9e9674c2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:37.395736 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:37.483172 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 560/1123] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Sep 2 09:15:37 2022] PUT /identity/v3/projects/60b1fc1dd5114cf3bf82b38cb201320f/users/0f69025973b24e1f8a869932ce5e235e/roles/f2fe9fb31024471e9e2bea2b12715de6 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:39.341660 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72df90-39cb-4b74-a733-806d379c8e27 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:39.342170 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72df90-39cb-4b74-a733-806d379c8e27 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:39.342170 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72df90-39cb-4b74-a733-806d379c8e27 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:39.343492 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 564/1124] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:39.348688 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751c491-c39f-49bd-9668-0d9cdc10f170 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:39.349136 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751c491-c39f-49bd-9668-0d9cdc10f170 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:39.349298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c751c491-c39f-49bd-9668-0d9cdc10f170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:39.366960 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c751c491-c39f-49bd-9668-0d9cdc10f170 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:39.368137 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c751c491-c39f-49bd-9668-0d9cdc10f170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:39.431155 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 561/1125] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:15:39.701026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72df90-39cb-4b74-a733-806d379c8e27 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:39.753345 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:39.797973 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72df90-39cb-4b74-a733-806d379c8e27 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:39.831169 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ff628d-d604-444c-aedb-6c4a42e5692e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:39.831785 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff628d-d604-444c-aedb-6c4a42e5692e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:39.832067 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff628d-d604-444c-aedb-6c4a42e5692e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:39.832340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff628d-d604-444c-aedb-6c4a42e5692e None admin] PATH_INFO: `/v3/roles/f2fe9fb31024471e9e2bea2b12715de6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:39.833712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ff628d-d604-444c-aedb-6c4a42e5692e None admin] RBAC: Authorizing `identity:get_role(role_id=f2fe9fb31024471e9e2bea2b12715de6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:39.835079 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ff628d-d604-444c-aedb-6c4a42e5692e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:39.836369 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 565/1126] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:15:39 2022] GET /identity/v3/roles/f2fe9fb31024471e9e2bea2b12715de6 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:39.842031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c751c491-c39f-49bd-9668-0d9cdc10f170 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:39.938496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c751c491-c39f-49bd-9668-0d9cdc10f170 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:39.962329 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:39.973760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a0e09-8ad9-4a3c-9aa6-95fb9a7f0a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:39.974393 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a0e09-8ad9-4a3c-9aa6-95fb9a7f0a8b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:39.974675 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a0e09-8ad9-4a3c-9aa6-95fb9a7f0a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:39.974985 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a0e09-8ad9-4a3c-9aa6-95fb9a7f0a8b None admin] PATH_INFO: `/v3/users/0f69025973b24e1f8a869932ce5e235e` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:39.976102 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a0e09-8ad9-4a3c-9aa6-95fb9a7f0a8b None admin] RBAC: Authorizing `identity:get_user(user_id=0f69025973b24e1f8a869932ce5e235e)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:39.977574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a0e09-8ad9-4a3c-9aa6-95fb9a7f0a8b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:39.979178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 562/1127] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:15:39 2022] GET /identity/v3/users/0f69025973b24e1f8a869932ce5e235e => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:39.984918 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ff628d-d604-444c-aedb-6c4a42e5692e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:40.081780 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ff628d-d604-444c-aedb-6c4a42e5692e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:40.101120 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:40.114482 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55430d1-2169-43c0-b8bb-24bb5538f09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:40.115337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55430d1-2169-43c0-b8bb-24bb5538f09a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:40.115539 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55430d1-2169-43c0-b8bb-24bb5538f09a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:40.115662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55430d1-2169-43c0-b8bb-24bb5538f09a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:40.180514 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55430d1-2169-43c0-b8bb-24bb5538f09a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:40.181945 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55430d1-2169-43c0-b8bb-24bb5538f09a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:40.277543 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 566/1128] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:15:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 09:15:40.283359 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a0e09-8ad9-4a3c-9aa6-95fb9a7f0a8b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:40.383241 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a0e09-8ad9-4a3c-9aa6-95fb9a7f0a8b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:40.415427 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea79d5a-33d3-48b2-8c35-e3829fe972be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:40.416053 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea79d5a-33d3-48b2-8c35-e3829fe972be None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:40.416334 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea79d5a-33d3-48b2-8c35-e3829fe972be None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:40.416608 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea79d5a-33d3-48b2-8c35-e3829fe972be None admin] PATH_INFO: `/v3/projects/60b1fc1dd5114cf3bf82b38cb201320f` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:40.417715 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea79d5a-33d3-48b2-8c35-e3829fe972be None admin] RBAC: Authorizing `identity:get_project(project_id=60b1fc1dd5114cf3bf82b38cb201320f)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:40.419188 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea79d5a-33d3-48b2-8c35-e3829fe972be None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:40.420790 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 563/1129] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:15:40 2022] GET /identity/v3/projects/60b1fc1dd5114cf3bf82b38cb201320f => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:40.427403 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55430d1-2169-43c0-b8bb-24bb5538f09a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:40.448592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:40.523932 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55430d1-2169-43c0-b8bb-24bb5538f09a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:40.556346 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32101d-409a-4ef7-9417-38e10b06d569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:40.556880 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32101d-409a-4ef7-9417-38e10b06d569 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:40.557155 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32101d-409a-4ef7-9417-38e10b06d569 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:40.557424 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32101d-409a-4ef7-9417-38e10b06d569 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:40.558133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32101d-409a-4ef7-9417-38e10b06d569 None admin] RBAC: Adding query filter params (role.id=f2fe9fb31024471e9e2bea2b12715de6, scope.project.id=60b1fc1dd5114cf3bf82b38cb201320f, user.id=0f69025973b24e1f8a869932ce5e235e) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:40.558437 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32101d-409a-4ef7-9417-38e10b06d569 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:40.559872 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32101d-409a-4ef7-9417-38e10b06d569 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:40.565450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 567/1130] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:15:40 2022] GET /identity/v3/role_assignments?user.id=0f69025973b24e1f8a869932ce5e235e&scope.project.id=60b1fc1dd5114cf3bf82b38cb201320f&role.id=f2fe9fb31024471e9e2bea2b12715de6 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:42.478063 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8cee30-addd-4c1d-96fd-72fde50735df None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:42.478602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8cee30-addd-4c1d-96fd-72fde50735df None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:42.478602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8cee30-addd-4c1d-96fd-72fde50735df None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:42.480001 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 564/1131] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:42.485213 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9c5e8-d8aa-4df3-aec2-4706d332301d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:42.485577 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9c5e8-d8aa-4df3-aec2-4706d332301d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:42.485764 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9c5e8-d8aa-4df3-aec2-4706d332301d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:42.504090 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d4b9c5e8-d8aa-4df3-aec2-4706d332301d None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:42.505301 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d4b9c5e8-d8aa-4df3-aec2-4706d332301d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:42.570423 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 568/1132] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:15:42.838527 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8cee30-addd-4c1d-96fd-72fde50735df None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:42.897607 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:42.936106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8cee30-addd-4c1d-96fd-72fde50735df None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:42.970222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d15e790-fa65-4b06-9ca9-5047a3d52754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:42.971382 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d15e790-fa65-4b06-9ca9-5047a3d52754 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:42.971666 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d15e790-fa65-4b06-9ca9-5047a3d52754 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:42.971941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d15e790-fa65-4b06-9ca9-5047a3d52754 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:42.973045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d15e790-fa65-4b06-9ca9-5047a3d52754 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:42.974464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d15e790-fa65-4b06-9ca9-5047a3d52754 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:42.976066 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 565/1133] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:15:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:42.981918 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9c5e8-d8aa-4df3-aec2-4706d332301d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:42.997195 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:43.078081 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9c5e8-d8aa-4df3-aec2-4706d332301d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:43.110566 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75898a5-5553-4742-b867-240bab06fa7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:43.111836 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75898a5-5553-4742-b867-240bab06fa7a None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:43.112112 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75898a5-5553-4742-b867-240bab06fa7a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:43.112380 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75898a5-5553-4742-b867-240bab06fa7a None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:43.113020 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75898a5-5553-4742-b867-240bab06fa7a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:43.114430 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75898a5-5553-4742-b867-240bab06fa7a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:43.219122 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 569/1134] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Sep 2 09:15:42 2022] POST /identity/v3/projects => generated 311 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:15:45.119372 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f621005a-31ae-4e1b-b7fa-ec49a4bbd632 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:45.119866 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f621005a-31ae-4e1b-b7fa-ec49a4bbd632 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:45.119866 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f621005a-31ae-4e1b-b7fa-ec49a4bbd632 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:45.121187 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 566/1135] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:45.126516 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5155b-675a-4ee5-8487-ec4d2baba47b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:45.126951 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5155b-675a-4ee5-8487-ec4d2baba47b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:45.127106 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5155b-675a-4ee5-8487-ec4d2baba47b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:45.144815 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7ca5155b-675a-4ee5-8487-ec4d2baba47b None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:45.145991 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7ca5155b-675a-4ee5-8487-ec4d2baba47b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:45.203293 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:45.210544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 570/1136] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:15:45.478547 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f621005a-31ae-4e1b-b7fa-ec49a4bbd632 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:45.576373 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f621005a-31ae-4e1b-b7fa-ec49a4bbd632 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:45.601489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:45.608960 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4eb57-1e7a-432a-ae52-7d681d4b55c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:45.609577 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4eb57-1e7a-432a-ae52-7d681d4b55c2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:45.609857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4eb57-1e7a-432a-ae52-7d681d4b55c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:45.610130 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4eb57-1e7a-432a-ae52-7d681d4b55c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:45.611752 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4eb57-1e7a-432a-ae52-7d681d4b55c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:45.613160 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4eb57-1e7a-432a-ae52-7d681d4b55c2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:45.614805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 567/1137] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:15:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:45.620492 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca5155b-675a-4ee5-8487-ec4d2baba47b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:45.717503 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca5155b-675a-4ee5-8487-ec4d2baba47b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:45.750358 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1063504-26da-4bb5-92e7-81d991029866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:45.750929 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1063504-26da-4bb5-92e7-81d991029866 None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:45.751212 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1063504-26da-4bb5-92e7-81d991029866 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:45.752112 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1063504-26da-4bb5-92e7-81d991029866 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:45.752765 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1063504-26da-4bb5-92e7-81d991029866 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:45.754184 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1063504-26da-4bb5-92e7-81d991029866 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:45.837231 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 571/1138] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Sep 2 09:15:45 2022] POST /identity/v3/users => generated 286 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:15:47.756418 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d623a-6113-4511-a14a-6ec04fbdbaed None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:47.756973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d623a-6113-4511-a14a-6ec04fbdbaed None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:47.756973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d623a-6113-4511-a14a-6ec04fbdbaed None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:47.758309 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 568/1139] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:47.763637 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d0ce4-55cd-4a22-94bf-d3fe4e410ee0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:47.764023 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d0ce4-55cd-4a22-94bf-d3fe4e410ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:47.764190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399d0ce4-55cd-4a22-94bf-d3fe4e410ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:47.781829 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-399d0ce4-55cd-4a22-94bf-d3fe4e410ee0 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:47.783064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-399d0ce4-55cd-4a22-94bf-d3fe4e410ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:47.828775 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:47.847518 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 572/1140] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 09:15:48.116657 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d623a-6113-4511-a14a-6ec04fbdbaed None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:48.213492 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d623a-6113-4511-a14a-6ec04fbdbaed None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:48.246085 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c9beb-ccbb-4229-adc1-308276356aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:48.246691 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c9beb-ccbb-4229-adc1-308276356aa2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:48.246996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c9beb-ccbb-4229-adc1-308276356aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:48.247265 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52c9beb-ccbb-4229-adc1-308276356aa2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:48.256862 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52c9beb-ccbb-4229-adc1-308276356aa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:15:48.261531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-c52c9beb-ccbb-4229-adc1-308276356aa2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:15:48.261531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:48.261531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:48.261531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:48.261531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:48.261531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:48.261531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:48.261531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:48.261531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:48.261531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:48.261531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:48.261531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:48.261531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:15:48.261531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:15:48.261531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:15:48.261531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:15:48.262780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:15:48.262780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:15:48.262780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:15:48.262780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:15:48.262780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:48.262780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:48.262780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:48.262780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:48.262780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:48.262780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:48.262780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:48.262780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:48.262780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:48.262780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:48.262780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:48.262780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:48.264084 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:48.264084 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:48.264084 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:48.264084 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:48.264084 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:15:48.264084 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:15:48.264084 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:15:48.264084 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:15:48.264084 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:15:48.264084 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:15:48.264084 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:15:48.264084 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:48.264084 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 569/1141] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:15:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:48.268382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d0ce4-55cd-4a22-94bf-d3fe4e410ee0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:48.366052 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399d0ce4-55cd-4a22-94bf-d3fe4e410ee0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:48.397844 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfe69b-aa5d-48d3-826e-e8d62eb3ac07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:48.398386 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfe69b-aa5d-48d3-826e-e8d62eb3ac07 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:48.398663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfe69b-aa5d-48d3-826e-e8d62eb3ac07 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:48.398961 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfe69b-aa5d-48d3-826e-e8d62eb3ac07 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:48.399619 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfe69b-aa5d-48d3-826e-e8d62eb3ac07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:48.399919 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfe69b-aa5d-48d3-826e-e8d62eb3ac07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:48.401264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfe69b-aa5d-48d3-826e-e8d62eb3ac07 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:48.409453 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 573/1142] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:15:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:48.415279 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c9beb-ccbb-4229-adc1-308276356aa2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:48.469151 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:48.511212 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52c9beb-ccbb-4229-adc1-308276356aa2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:48.543841 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7120913-49dc-41d0-b964-9422bfa249ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:48.544445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7120913-49dc-41d0-b964-9422bfa249ad None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:48.544721 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7120913-49dc-41d0-b964-9422bfa249ad None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:48.544989 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7120913-49dc-41d0-b964-9422bfa249ad None admin] PATH_INFO: `/v3/users/6aa943b055e946b49dae80c4b4a04c2e` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:48.557325 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7120913-49dc-41d0-b964-9422bfa249ad None admin] RBAC: Authorizing `identity:get_user(user_id=6aa943b055e946b49dae80c4b4a04c2e)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:48.558753 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7120913-49dc-41d0-b964-9422bfa249ad None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:48.560296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 570/1143] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:15:48 2022] GET /identity/v3/users/6aa943b055e946b49dae80c4b4a04c2e => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:48.566115 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfe69b-aa5d-48d3-826e-e8d62eb3ac07 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:48.611916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:48.665348 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfe69b-aa5d-48d3-826e-e8d62eb3ac07 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:48.697166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb8d8f-8e26-4dd8-a440-235263e277c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:48.697762 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb8d8f-8e26-4dd8-a440-235263e277c3 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:48.698042 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb8d8f-8e26-4dd8-a440-235263e277c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:48.698310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb8d8f-8e26-4dd8-a440-235263e277c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:48.762024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb8d8f-8e26-4dd8-a440-235263e277c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:48.763483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb8d8f-8e26-4dd8-a440-235263e277c3 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:48.805768 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:48.858760 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 574/1144] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:15:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 09:15:48.864787 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7120913-49dc-41d0-b964-9422bfa249ad None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:48.959861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7120913-49dc-41d0-b964-9422bfa249ad None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:48.973739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:48.993330 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca034f0-209e-4f08-938d-c40cdc9eb67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:48.994222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca034f0-209e-4f08-938d-c40cdc9eb67d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:48.994510 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca034f0-209e-4f08-938d-c40cdc9eb67d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:48.994816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca034f0-209e-4f08-938d-c40cdc9eb67d None admin] PATH_INFO: `/v3/projects/1bf7c805269a429b8a433972094a6bba` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:48.995951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca034f0-209e-4f08-938d-c40cdc9eb67d None admin] RBAC: Authorizing `identity:get_project(project_id=1bf7c805269a429b8a433972094a6bba)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:48.997368 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca034f0-209e-4f08-938d-c40cdc9eb67d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:48.999021 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 571/1145] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:15:48 2022] GET /identity/v3/projects/1bf7c805269a429b8a433972094a6bba => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:49.005269 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb8d8f-8e26-4dd8-a440-235263e277c3 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:49.099914 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb8d8f-8e26-4dd8-a440-235263e277c3 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:49.133012 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11604d0c-90cb-4154-bdb0-aadb9d42504e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:49.133554 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11604d0c-90cb-4154-bdb0-aadb9d42504e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:49.133835 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11604d0c-90cb-4154-bdb0-aadb9d42504e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:49.134107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11604d0c-90cb-4154-bdb0-aadb9d42504e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:49.134869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11604d0c-90cb-4154-bdb0-aadb9d42504e None admin] RBAC: Adding query filter params (role.id=f886b8e8889549e896a8fb3426cce9e1, scope.project.id=1bf7c805269a429b8a433972094a6bba, user.id=6aa943b055e946b49dae80c4b4a04c2e) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:49.135181 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11604d0c-90cb-4154-bdb0-aadb9d42504e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:49.136602 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11604d0c-90cb-4154-bdb0-aadb9d42504e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:49.141842 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 575/1146] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:15:49 2022] GET /identity/v3/role_assignments?user.id=6aa943b055e946b49dae80c4b4a04c2e&scope.project.id=1bf7c805269a429b8a433972094a6bba&role.id=f886b8e8889549e896a8fb3426cce9e1 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:50.982391 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bb0da-c3dc-48db-bd0d-1fdca4a41eda None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:50.982940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bb0da-c3dc-48db-bd0d-1fdca4a41eda None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:50.983039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5bb0da-c3dc-48db-bd0d-1fdca4a41eda None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:50.984312 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 572/1147] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:50.989920 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeeaad3-15c4-478e-9777-a6d40c3e56b9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:50.990300 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeeaad3-15c4-478e-9777-a6d40c3e56b9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:50.990457 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeeaad3-15c4-478e-9777-a6d40c3e56b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:51.008719 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-caeeaad3-15c4-478e-9777-a6d40c3e56b9 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:51.009891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-caeeaad3-15c4-478e-9777-a6d40c3e56b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:51.011865 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:51.075968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 576/1148] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 09:15:51.337796 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5bb0da-c3dc-48db-bd0d-1fdca4a41eda None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:51.436792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5bb0da-c3dc-48db-bd0d-1fdca4a41eda None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:51.468844 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a80d5-0a32-49ba-9533-a4a6303655d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:51.469454 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a80d5-0a32-49ba-9533-a4a6303655d0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:51.470222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a80d5-0a32-49ba-9533-a4a6303655d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:51.470495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6a80d5-0a32-49ba-9533-a4a6303655d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:51.480219 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6a80d5-0a32-49ba-9533-a4a6303655d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:15:51.484919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-bf6a80d5-0a32-49ba-9533-a4a6303655d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:15:51.484919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:51.484919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:51.484919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:51.484919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:51.484919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:51.484919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:51.484919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:51.484919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:51.484919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:51.484919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:51.484919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:51.484919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:15:51.484919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:15:51.484919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:15:51.484919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:15:51.486679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:15:51.486679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:15:51.486679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:15:51.486679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:15:51.486679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:51.486679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:51.486679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:51.486679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:51.486679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:51.486679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:51.486679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:51.486679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:51.486679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:51.486679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:51.486679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:51.486679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:51.487612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:51.487612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:51.487612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:51.487612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:51.487612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:15:51.487612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:15:51.487612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:15:51.487612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:15:51.487612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:15:51.487612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:15:51.487612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:15:51.487612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:51.487612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 573/1149] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:15:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:51.492527 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeeaad3-15c4-478e-9777-a6d40c3e56b9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:51.588337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeeaad3-15c4-478e-9777-a6d40c3e56b9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:51.621083 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973f892-1fb4-400e-8c3c-5dad1c78aa25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:51.621670 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973f892-1fb4-400e-8c3c-5dad1c78aa25 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:51.621946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973f892-1fb4-400e-8c3c-5dad1c78aa25 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:51.622214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c973f892-1fb4-400e-8c3c-5dad1c78aa25 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:51.622894 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c973f892-1fb4-400e-8c3c-5dad1c78aa25 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:51.623194 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c973f892-1fb4-400e-8c3c-5dad1c78aa25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:51.624536 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c973f892-1fb4-400e-8c3c-5dad1c78aa25 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:51.629514 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:51.632775 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 577/1150] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:15:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:51.638728 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a80d5-0a32-49ba-9533-a4a6303655d0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:51.682258 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:51.737418 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6a80d5-0a32-49ba-9533-a4a6303655d0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:51.770861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab1ee1-5dd2-47ab-a898-94dca09ceef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:51.771506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab1ee1-5dd2-47ab-a898-94dca09ceef0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:51.771782 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab1ee1-5dd2-47ab-a898-94dca09ceef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:51.772051 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab1ee1-5dd2-47ab-a898-94dca09ceef0 None admin] PATH_INFO: `/v3/users/6aa943b055e946b49dae80c4b4a04c2e` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:51.773145 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab1ee1-5dd2-47ab-a898-94dca09ceef0 None admin] RBAC: Authorizing `identity:get_user(user_id=6aa943b055e946b49dae80c4b4a04c2e)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:51.774554 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab1ee1-5dd2-47ab-a898-94dca09ceef0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:51.776122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 574/1151] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:15:51 2022] GET /identity/v3/users/6aa943b055e946b49dae80c4b4a04c2e => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:51.781928 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973f892-1fb4-400e-8c3c-5dad1c78aa25 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:51.879168 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c973f892-1fb4-400e-8c3c-5dad1c78aa25 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:51.911875 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a310a-6df6-46ab-8053-3d8dfd9380a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:51.912501 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a310a-6df6-46ab-8053-3d8dfd9380a9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:51.912781 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a310a-6df6-46ab-8053-3d8dfd9380a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:51.913054 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a310a-6df6-46ab-8053-3d8dfd9380a9 None admin] PATH_INFO: `/v3/projects/1bf7c805269a429b8a433972094a6bba` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:51.914168 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a310a-6df6-46ab-8053-3d8dfd9380a9 None admin] RBAC: Authorizing `identity:get_project(project_id=1bf7c805269a429b8a433972094a6bba)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:51.915635 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a310a-6df6-46ab-8053-3d8dfd9380a9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:51.917258 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 578/1152] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:15:51 2022] GET /identity/v3/projects/1bf7c805269a429b8a433972094a6bba => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:51.923359 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab1ee1-5dd2-47ab-a898-94dca09ceef0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:52.020321 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab1ee1-5dd2-47ab-a898-94dca09ceef0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:52.030587 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:52.053013 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf04ba-2e33-4475-b93c-d0ac68e1eca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:52.053658 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf04ba-2e33-4475-b93c-d0ac68e1eca8 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:52.053939 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf04ba-2e33-4475-b93c-d0ac68e1eca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:52.054213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf04ba-2e33-4475-b93c-d0ac68e1eca8 None admin] PATH_INFO: `/v3/projects/1bf7c805269a429b8a433972094a6bba/users/6aa943b055e946b49dae80c4b4a04c2e/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:52.056166 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf04ba-2e33-4475-b93c-d0ac68e1eca8 None admin] RBAC: Authorizing `identity:create_grant(project_id=1bf7c805269a429b8a433972094a6bba, user_id=6aa943b055e946b49dae80c4b4a04c2e, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:52.060690 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bf04ba-2e33-4475-b93c-d0ac68e1eca8 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:52.139078 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 575/1153] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Sep 2 09:15:51 2022] PUT /identity/v3/projects/1bf7c805269a429b8a433972094a6bba/users/6aa943b055e946b49dae80c4b4a04c2e/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:15:54.002554 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a87a3-61bf-476c-9fec-6ad660a385e1 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:54.003127 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a87a3-61bf-476c-9fec-6ad660a385e1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:54.003127 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685a87a3-61bf-476c-9fec-6ad660a385e1 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:54.004480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 579/1154] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:54.009707 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90f2e4-2451-46fe-b9ed-24b125247970 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:54.010082 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90f2e4-2451-46fe-b9ed-24b125247970 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:54.010256 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca90f2e4-2451-46fe-b9ed-24b125247970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:54.028457 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ca90f2e4-2451-46fe-b9ed-24b125247970 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:54.029654 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ca90f2e4-2451-46fe-b9ed-24b125247970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:54.095807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 576/1155] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:15:54.369645 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a87a3-61bf-476c-9fec-6ad660a385e1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:54.436285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:54.470814 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685a87a3-61bf-476c-9fec-6ad660a385e1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:54.504092 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2db99-888c-4ea3-ae46-42ef0e38cd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:54.504706 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2db99-888c-4ea3-ae46-42ef0e38cd68 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:54.504983 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2db99-888c-4ea3-ae46-42ef0e38cd68 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:54.505251 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2db99-888c-4ea3-ae46-42ef0e38cd68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:54.515451 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2db99-888c-4ea3-ae46-42ef0e38cd68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:15:54.520165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-b7a2db99-888c-4ea3-ae46-42ef0e38cd68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:15:54.520165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:54.520165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:54.520165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:54.520165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:54.520165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:54.520165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:54.520165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:54.520165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:54.520165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:54.520165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:54.520165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:54.520165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:15:54.520165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:15:54.520165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:15:54.520165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:15:54.521278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:15:54.521278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:15:54.521278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:15:54.521278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:15:54.521278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:54.521278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:54.521278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:54.521278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:54.521278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:54.521278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:54.521278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:54.521278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:54.521278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:54.521278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:54.521278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:54.521278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:54.522393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:54.522393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:54.522393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:54.522393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:54.522393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:15:54.522393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:15:54.522393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:15:54.522393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:15:54.522393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:15:54.522393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:15:54.522393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:15:54.522393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:15:54.522393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 580/1156] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:15:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:54.527390 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90f2e4-2451-46fe-b9ed-24b125247970 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:54.620414 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:54.623145 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca90f2e4-2451-46fe-b9ed-24b125247970 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:54.658540 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b0341-06a8-4951-a761-c228e915c6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:54.659224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b0341-06a8-4951-a761-c228e915c6ac None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:54.659505 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b0341-06a8-4951-a761-c228e915c6ac None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:54.659776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b0341-06a8-4951-a761-c228e915c6ac None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:54.660426 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623b0341-06a8-4951-a761-c228e915c6ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:54.660759 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623b0341-06a8-4951-a761-c228e915c6ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:54.662105 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623b0341-06a8-4951-a761-c228e915c6ac None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:54.669837 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 577/1157] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:15:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:54.675575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2db99-888c-4ea3-ae46-42ef0e38cd68 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:54.772164 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2db99-888c-4ea3-ae46-42ef0e38cd68 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:54.797370 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:54.804717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cab103-d830-4585-b867-59b783b4a413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:54.805332 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cab103-d830-4585-b867-59b783b4a413 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:54.805614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cab103-d830-4585-b867-59b783b4a413 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:54.805888 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cab103-d830-4585-b867-59b783b4a413 None admin] PATH_INFO: `/v3/users/6aa943b055e946b49dae80c4b4a04c2e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:54.807026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cab103-d830-4585-b867-59b783b4a413 None admin] RBAC: Authorizing `identity:get_user(user_id=6aa943b055e946b49dae80c4b4a04c2e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:54.808438 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cab103-d830-4585-b867-59b783b4a413 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:54.810026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 581/1158] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:15:54 2022] GET /identity/v3/users/6aa943b055e946b49dae80c4b4a04c2e => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:54.816535 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b0341-06a8-4951-a761-c228e915c6ac None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:54.914586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b0341-06a8-4951-a761-c228e915c6ac None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:54.946846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f519c-0919-4719-bb2a-38eeb9b273f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:54.947440 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f519c-0919-4719-bb2a-38eeb9b273f6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:54.947717 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f519c-0919-4719-bb2a-38eeb9b273f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:54.947990 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f519c-0919-4719-bb2a-38eeb9b273f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:54.977010 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:55.012886 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4f519c-0919-4719-bb2a-38eeb9b273f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:55.014317 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4f519c-0919-4719-bb2a-38eeb9b273f6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:55.107937 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 578/1159] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:15:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 09:15:55.114396 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cab103-d830-4585-b867-59b783b4a413 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:55.211170 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cab103-d830-4585-b867-59b783b4a413 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:55.243407 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9843e06-457a-41bf-81e4-ad3d6c1b275d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:55.244027 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9843e06-457a-41bf-81e4-ad3d6c1b275d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:55.244308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9843e06-457a-41bf-81e4-ad3d6c1b275d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:55.244583 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9843e06-457a-41bf-81e4-ad3d6c1b275d None admin] PATH_INFO: `/v3/projects/1bf7c805269a429b8a433972094a6bba` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:55.245688 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9843e06-457a-41bf-81e4-ad3d6c1b275d None admin] RBAC: Authorizing `identity:get_project(project_id=1bf7c805269a429b8a433972094a6bba)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:55.247152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9843e06-457a-41bf-81e4-ad3d6c1b275d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:55.248763 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 582/1160] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:15:55 2022] GET /identity/v3/projects/1bf7c805269a429b8a433972094a6bba => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:55.254634 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f519c-0919-4719-bb2a-38eeb9b273f6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:55.313450 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:55.350247 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f519c-0919-4719-bb2a-38eeb9b273f6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:55.384004 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760e194-6d6f-4656-b464-755b0a79daae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:55.384589 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760e194-6d6f-4656-b464-755b0a79daae None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:55.384866 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760e194-6d6f-4656-b464-755b0a79daae None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:55.385138 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760e194-6d6f-4656-b464-755b0a79daae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:55.385849 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f760e194-6d6f-4656-b464-755b0a79daae None admin] RBAC: Adding query filter params (role.id=f886b8e8889549e896a8fb3426cce9e1, scope.project.id=1bf7c805269a429b8a433972094a6bba, user.id=6aa943b055e946b49dae80c4b4a04c2e) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:55.386155 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f760e194-6d6f-4656-b464-755b0a79daae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:55.387602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f760e194-6d6f-4656-b464-755b0a79daae None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:55.393094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 579/1161] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:15:55 2022] GET /identity/v3/role_assignments?user.id=6aa943b055e946b49dae80c4b4a04c2e&scope.project.id=1bf7c805269a429b8a433972094a6bba&role.id=f886b8e8889549e896a8fb3426cce9e1 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:15:57.290407 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd36de-3fc4-498e-9237-3c4d0798eb80 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:57.290978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd36de-3fc4-498e-9237-3c4d0798eb80 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:57.290978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd36de-3fc4-498e-9237-3c4d0798eb80 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:57.292299 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 583/1162] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:15:57.297306 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30207b4-ec4b-4cba-b9f6-73abfd33782c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:57.297677 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30207b4-ec4b-4cba-b9f6-73abfd33782c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:57.297850 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30207b4-ec4b-4cba-b9f6-73abfd33782c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:57.316571 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f30207b4-ec4b-4cba-b9f6-73abfd33782c None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:15:57.318004 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f30207b4-ec4b-4cba-b9f6-73abfd33782c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:15:57.381481 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 580/1163] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:15:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:15:57.648625 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd36de-3fc4-498e-9237-3c4d0798eb80 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:57.710797 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:57.748103 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd36de-3fc4-498e-9237-3c4d0798eb80 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:57.780069 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596dbadf-a343-4f3b-b859-008d0cc36586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:57.780677 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596dbadf-a343-4f3b-b859-008d0cc36586 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:57.780954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596dbadf-a343-4f3b-b859-008d0cc36586 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:57.781226 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596dbadf-a343-4f3b-b859-008d0cc36586 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:57.848603 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596dbadf-a343-4f3b-b859-008d0cc36586 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:57.850037 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596dbadf-a343-4f3b-b859-008d0cc36586 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:57.908006 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:57.945164 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 584/1164] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:15:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 09:15:57.950751 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30207b4-ec4b-4cba-b9f6-73abfd33782c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:58.004303 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:15:58.045621 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30207b4-ec4b-4cba-b9f6-73abfd33782c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:58.078326 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8a537-6ca1-4221-8e30-410d7df5f368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:58.078967 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8a537-6ca1-4221-8e30-410d7df5f368 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:58.079244 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8a537-6ca1-4221-8e30-410d7df5f368 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:58.079513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8a537-6ca1-4221-8e30-410d7df5f368 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:58.085865 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e8a537-6ca1-4221-8e30-410d7df5f368 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:58.087313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e8a537-6ca1-4221-8e30-410d7df5f368 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:58.098760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-71e8a537-6ca1-4221-8e30-410d7df5f368 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 02 09:15:58.098760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:58.098760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 02 09:15:58.098760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 02 09:15:58.098760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:15:58.098760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:15:58.098760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:15:58.098760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:15:58.098760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 02 09:15:58.098760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:58.098760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:15:58.098760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:58.098760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:15:58.098760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:15:58.098760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:15:58.098760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:15:58.098760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:15:58.100079 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:15:58.100079 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:15:58.100079 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:15:58.100079 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:15:58.100079 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:15:58.100079 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:15:58.100079 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:15:58.100079 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:15:58.100079 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:15:58.100079 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:15:58.100079 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:15:58.100079 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:15:58.100079 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:15:58.100079 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:15:58.100079 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:15:58.100079 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:15:58.101269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:15:58.101269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:15:58.101269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:15:58.101269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:15:58.101269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:15:58.101269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:15:58.101269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:15:58.101269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:15:58.101269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:15:58.101269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:15:58.101269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 02 09:15:58.101269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:15:58.101269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 02 09:15:58.101269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:15:58.101269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 581/1165] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Sep 2 09:15:57 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:15:58.105827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596dbadf-a343-4f3b-b859-008d0cc36586 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:15:58.201449 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596dbadf-a343-4f3b-b859-008d0cc36586 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:15:58.233838 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5394266-ebb4-4bed-9fb5-217f09deb51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:15:58.234424 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5394266-ebb4-4bed-9fb5-217f09deb51f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:15:58.234699 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5394266-ebb4-4bed-9fb5-217f09deb51f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:15:58.235000 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5394266-ebb4-4bed-9fb5-217f09deb51f None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:15:58.236145 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5394266-ebb4-4bed-9fb5-217f09deb51f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:15:58.236446 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5394266-ebb4-4bed-9fb5-217f09deb51f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:15:58.237784 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5394266-ebb4-4bed-9fb5-217f09deb51f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:15:58.244613 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 585/1166] 10.222.0.57 () {56 vars in 1115 bytes} [Fri Sep 2 09:15:58 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:16:00.116574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707794cd-14e6-40d8-9b6d-218bbc5b1b42 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:00.117063 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707794cd-14e6-40d8-9b6d-218bbc5b1b42 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:00.117063 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707794cd-14e6-40d8-9b6d-218bbc5b1b42 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:00.118413 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 582/1167] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:16:00.123551 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae915d5c-a772-4704-8287-891056a0d40c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:00.124934 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae915d5c-a772-4704-8287-891056a0d40c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:00.124934 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae915d5c-a772-4704-8287-891056a0d40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:00.141740 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ae915d5c-a772-4704-8287-891056a0d40c None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:00.142960 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ae915d5c-a772-4704-8287-891056a0d40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:00.165186 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:00.207584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 586/1168] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:16:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:16:00.471647 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707794cd-14e6-40d8-9b6d-218bbc5b1b42 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:00.569273 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707794cd-14e6-40d8-9b6d-218bbc5b1b42 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:00.584207 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:00.600723 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09a23c-6dd2-49bc-9a41-8fc46df23305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:00.601316 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09a23c-6dd2-49bc-9a41-8fc46df23305 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:00.601592 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09a23c-6dd2-49bc-9a41-8fc46df23305 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:00.601862 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09a23c-6dd2-49bc-9a41-8fc46df23305 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:00.602750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09a23c-6dd2-49bc-9a41-8fc46df23305 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:00.604108 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09a23c-6dd2-49bc-9a41-8fc46df23305 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:00.694423 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 583/1169] 10.222.0.57 () {60 vars in 1138 bytes} [Fri Sep 2 09:16:00 2022] POST /identity/v3/domains => generated 267 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:02.593736 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19072f1-dea4-4e8f-a9c7-a0af39849d54 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:02.594290 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19072f1-dea4-4e8f-a9c7-a0af39849d54 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:02.594290 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19072f1-dea4-4e8f-a9c7-a0af39849d54 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:02.596179 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 587/1170] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:16:02.601420 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f05de-c9c0-4969-8778-1931ef207e8b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:02.601835 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f05de-c9c0-4969-8778-1931ef207e8b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:02.601976 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f05de-c9c0-4969-8778-1931ef207e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:02.620308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-434f05de-c9c0-4969-8778-1931ef207e8b None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:02.621488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-434f05de-c9c0-4969-8778-1931ef207e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:02.685020 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 584/1171] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:16:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:16:02.956494 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19072f1-dea4-4e8f-a9c7-a0af39849d54 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:03.055649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19072f1-dea4-4e8f-a9c7-a0af39849d54 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:03.088909 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95976117-89b2-4b63-8e4e-586a4206166d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:03.089527 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95976117-89b2-4b63-8e4e-586a4206166d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:03.089801 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95976117-89b2-4b63-8e4e-586a4206166d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:03.090064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95976117-89b2-4b63-8e4e-586a4206166d None admin] PATH_INFO: `/v3/domains/38c5677fb853476498fc56ad85512adf` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:03.100033 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95976117-89b2-4b63-8e4e-586a4206166d None admin] RBAC: Authorizing `identity:get_domain(domain_id=38c5677fb853476498fc56ad85512adf)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:03.101397 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95976117-89b2-4b63-8e4e-586a4206166d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:03.102993 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 588/1172] 10.222.0.57 () {56 vars in 1183 bytes} [Fri Sep 2 09:16:02 2022] GET /identity/v3/domains/38c5677fb853476498fc56ad85512adf => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:16:03.109008 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f05de-c9c0-4969-8778-1931ef207e8b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:03.203792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f05de-c9c0-4969-8778-1931ef207e8b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:03.212100 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:03.237870 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c917e-062f-4cdd-8661-e5762db8bff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:03.238490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c917e-062f-4cdd-8661-e5762db8bff5 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:03.238802 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c917e-062f-4cdd-8661-e5762db8bff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:03.239077 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c917e-062f-4cdd-8661-e5762db8bff5 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:03.239716 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c917e-062f-4cdd-8661-e5762db8bff5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:03.241155 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c917e-062f-4cdd-8661-e5762db8bff5 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:03.328824 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 585/1173] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Sep 2 09:16:03 2022] POST /identity/v3/projects => generated 361 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:16:05.244845 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127c7a4-d648-4bf9-95aa-d15cddee4371 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:05.245340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127c7a4-d648-4bf9-95aa-d15cddee4371 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:05.245340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127c7a4-d648-4bf9-95aa-d15cddee4371 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:05.246656 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 589/1174] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:16:05.252396 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95ced0-1f7c-4bdf-88bd-624d96de88ce None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:05.252767 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95ced0-1f7c-4bdf-88bd-624d96de88ce None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:05.252973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95ced0-1f7c-4bdf-88bd-624d96de88ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:05.270660 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-7c95ced0-1f7c-4bdf-88bd-624d96de88ce None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:05.271870 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7c95ced0-1f7c-4bdf-88bd-624d96de88ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:05.334889 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 586/1175] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:16:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:16:05.602275 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127c7a4-d648-4bf9-95aa-d15cddee4371 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:05.622041 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:05.701550 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127c7a4-d648-4bf9-95aa-d15cddee4371 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:05.733523 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a505e4c-5f88-4428-aac0-223e93277592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:05.734139 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a505e4c-5f88-4428-aac0-223e93277592 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:05.734415 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a505e4c-5f88-4428-aac0-223e93277592 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:05.734693 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a505e4c-5f88-4428-aac0-223e93277592 None admin] PATH_INFO: `/v3/domains/38c5677fb853476498fc56ad85512adf` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:05.736316 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a505e4c-5f88-4428-aac0-223e93277592 None admin] RBAC: Authorizing `identity:get_domain(domain_id=38c5677fb853476498fc56ad85512adf)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:05.737726 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a505e4c-5f88-4428-aac0-223e93277592 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:05.739301 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 590/1176] 10.222.0.57 () {56 vars in 1183 bytes} [Fri Sep 2 09:16:05 2022] GET /identity/v3/domains/38c5677fb853476498fc56ad85512adf => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:16:05.745002 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95ced0-1f7c-4bdf-88bd-624d96de88ce None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:05.828438 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:05.839708 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95ced0-1f7c-4bdf-88bd-624d96de88ce None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:05.872918 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd8008-5434-4c63-a3fe-3f561a432e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:05.873456 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd8008-5434-4c63-a3fe-3f561a432e2b None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:05.873736 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd8008-5434-4c63-a3fe-3f561a432e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:05.874010 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd8008-5434-4c63-a3fe-3f561a432e2b None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:05.874661 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfd8008-5434-4c63-a3fe-3f561a432e2b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:05.876762 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfd8008-5434-4c63-a3fe-3f561a432e2b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:05.982357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 587/1177] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Sep 2 09:16:05 2022] POST /identity/v3/users => generated 311 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:16:07.903220 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063bdc1-228e-42f3-98a5-2d54f0e23c3d None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:07.903750 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063bdc1-228e-42f3-98a5-2d54f0e23c3d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:07.903750 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063bdc1-228e-42f3-98a5-2d54f0e23c3d None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:07.905113 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 591/1178] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:16:07.910321 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8478cf-ef77-4f7c-84da-50743fcf03ed None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:07.910741 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8478cf-ef77-4f7c-84da-50743fcf03ed None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:07.910915 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8478cf-ef77-4f7c-84da-50743fcf03ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:07.928494 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-bf8478cf-ef77-4f7c-84da-50743fcf03ed None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:07.929675 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bf8478cf-ef77-4f7c-84da-50743fcf03ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:07.994196 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 588/1179] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:16:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:16:08.263430 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063bdc1-228e-42f3-98a5-2d54f0e23c3d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:08.345976 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:08.361353 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063bdc1-228e-42f3-98a5-2d54f0e23c3d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:08.393776 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9318cd1-d170-4d60-98b8-6a5cd791ad17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:08.394387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9318cd1-d170-4d60-98b8-6a5cd791ad17 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:08.394661 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9318cd1-d170-4d60-98b8-6a5cd791ad17 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:08.394955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9318cd1-d170-4d60-98b8-6a5cd791ad17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:08.404579 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9318cd1-d170-4d60-98b8-6a5cd791ad17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:16:08.409821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-a9318cd1-d170-4d60-98b8-6a5cd791ad17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:16:08.409821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:16:08.409821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:16:08.409821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:16:08.409821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:16:08.409821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:16:08.409821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:16:08.409821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:16:08.409821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:16:08.409821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:16:08.409821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:16:08.409821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:16:08.409821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:16:08.409821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:16:08.409821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:16:08.409821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:16:08.411030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:16:08.411030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:16:08.411030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:16:08.411030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:16:08.411030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:16:08.411030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:16:08.411030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:16:08.411030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:16:08.411030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:16:08.411030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:16:08.411030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:16:08.411030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:16:08.411030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:16:08.411030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:16:08.411030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:16:08.411030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:16:08.411931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:16:08.411931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:16:08.411931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:16:08.411931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:16:08.411931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:16:08.411931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:16:08.411931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:16:08.411931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:16:08.411931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:16:08.411931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:16:08.411931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:16:08.411931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:16:08.411931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 592/1180] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:16:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:16:08.416729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8478cf-ef77-4f7c-84da-50743fcf03ed None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:08.487714 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:08.510583 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8478cf-ef77-4f7c-84da-50743fcf03ed None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:08.543223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40664ad6-54c5-42bf-95f7-986d3493c58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:08.543845 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40664ad6-54c5-42bf-95f7-986d3493c58d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:08.544342 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40664ad6-54c5-42bf-95f7-986d3493c58d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:08.544791 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40664ad6-54c5-42bf-95f7-986d3493c58d None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:08.545869 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40664ad6-54c5-42bf-95f7-986d3493c58d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:16:08.546178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40664ad6-54c5-42bf-95f7-986d3493c58d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:08.548147 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40664ad6-54c5-42bf-95f7-986d3493c58d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:08.555999 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 589/1181] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:16:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:16:08.561494 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9318cd1-d170-4d60-98b8-6a5cd791ad17 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:08.735690 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9318cd1-d170-4d60-98b8-6a5cd791ad17 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:08.771243 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7706c-fdbe-4ede-aadf-87e0ae176d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:08.771889 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7706c-fdbe-4ede-aadf-87e0ae176d4e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:08.772128 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7706c-fdbe-4ede-aadf-87e0ae176d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:08.772396 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7706c-fdbe-4ede-aadf-87e0ae176d4e None admin] PATH_INFO: `/v3/users/eccec47b76704502ae0051f8fa121c0b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:08.785505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7706c-fdbe-4ede-aadf-87e0ae176d4e None admin] RBAC: Authorizing `identity:get_user(user_id=eccec47b76704502ae0051f8fa121c0b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:08.786955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7706c-fdbe-4ede-aadf-87e0ae176d4e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:08.788523 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 593/1182] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:16:08 2022] GET /identity/v3/users/eccec47b76704502ae0051f8fa121c0b => generated 311 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:16:08.794502 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40664ad6-54c5-42bf-95f7-986d3493c58d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:08.889160 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40664ad6-54c5-42bf-95f7-986d3493c58d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:08.921558 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b4e04-e6e2-4bfc-b256-bc214d564a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:08.922148 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b4e04-e6e2-4bfc-b256-bc214d564a71 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:08.922456 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b4e04-e6e2-4bfc-b256-bc214d564a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:08.922751 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b4e04-e6e2-4bfc-b256-bc214d564a71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:08.925957 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:08.986543 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b4e04-e6e2-4bfc-b256-bc214d564a71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:08.987992 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b4e04-e6e2-4bfc-b256-bc214d564a71 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:09.081056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 590/1183] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:16:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 09:16:09.086903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7706c-fdbe-4ede-aadf-87e0ae176d4e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:09.090611 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:09.182942 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7706c-fdbe-4ede-aadf-87e0ae176d4e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:09.215250 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e18d70-f7ab-4efa-a5b3-e85c652fc98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:09.215869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e18d70-f7ab-4efa-a5b3-e85c652fc98c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:09.216149 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e18d70-f7ab-4efa-a5b3-e85c652fc98c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:09.216418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e18d70-f7ab-4efa-a5b3-e85c652fc98c None admin] PATH_INFO: `/v3/projects/5171161ba19a42c994dc9b82d6ac5537` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:09.217517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e18d70-f7ab-4efa-a5b3-e85c652fc98c None admin] RBAC: Authorizing `identity:get_project(project_id=5171161ba19a42c994dc9b82d6ac5537)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:09.218958 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e18d70-f7ab-4efa-a5b3-e85c652fc98c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:09.221090 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 594/1184] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:16:09 2022] GET /identity/v3/projects/5171161ba19a42c994dc9b82d6ac5537 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:16:09.227118 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b4e04-e6e2-4bfc-b256-bc214d564a71 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:09.260300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:09.321817 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b4e04-e6e2-4bfc-b256-bc214d564a71 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:09.353847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed274d97-45be-454d-adf2-949afc383ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:09.354433 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed274d97-45be-454d-adf2-949afc383ebf None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:09.354735 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed274d97-45be-454d-adf2-949afc383ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:09.355007 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed274d97-45be-454d-adf2-949afc383ebf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:09.355722 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed274d97-45be-454d-adf2-949afc383ebf None admin] RBAC: Adding query filter params (role.id=f886b8e8889549e896a8fb3426cce9e1, scope.project.id=5171161ba19a42c994dc9b82d6ac5537, user.id=eccec47b76704502ae0051f8fa121c0b) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:16:09.356025 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed274d97-45be-454d-adf2-949afc383ebf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:09.357417 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed274d97-45be-454d-adf2-949afc383ebf None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:09.363080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 591/1185] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:16:09 2022] GET /identity/v3/role_assignments?user.id=eccec47b76704502ae0051f8fa121c0b&scope.project.id=5171161ba19a42c994dc9b82d6ac5537&role.id=f886b8e8889549e896a8fb3426cce9e1 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:16:11.241619 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eca9c0-c28d-4884-a4e9-583a5bb93083 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:11.242137 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eca9c0-c28d-4884-a4e9-583a5bb93083 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:11.242137 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eca9c0-c28d-4884-a4e9-583a5bb93083 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:11.243491 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 595/1186] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:16:11.248640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222c6b5-257c-437f-8540-4b250281c904 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:11.255948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222c6b5-257c-437f-8540-4b250281c904 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:11.255948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f222c6b5-257c-437f-8540-4b250281c904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:11.268054 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f222c6b5-257c-437f-8540-4b250281c904 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:11.269238 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f222c6b5-257c-437f-8540-4b250281c904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:11.335317 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 592/1187] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:16:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:16:11.610289 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eca9c0-c28d-4884-a4e9-583a5bb93083 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:11.678723 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:11.707466 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eca9c0-c28d-4884-a4e9-583a5bb93083 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:11.740524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef910d9-3017-47bf-bd4a-0900de17526e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:11.741140 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef910d9-3017-47bf-bd4a-0900de17526e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:11.741419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef910d9-3017-47bf-bd4a-0900de17526e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:11.741691 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef910d9-3017-47bf-bd4a-0900de17526e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:11.751416 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef910d9-3017-47bf-bd4a-0900de17526e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:16:11.756133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-5ef910d9-3017-47bf-bd4a-0900de17526e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:16:11.756133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:16:11.756133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:16:11.756133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:16:11.756133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:16:11.756133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:16:11.756133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:16:11.756133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:16:11.756133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:16:11.756133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:16:11.756133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:16:11.756133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:16:11.756133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:16:11.756133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:16:11.756133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:16:11.756133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:16:11.757236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:16:11.757236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:16:11.757236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:16:11.757236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:16:11.757236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:16:11.757236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:16:11.757236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:16:11.757236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:16:11.757236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:16:11.757236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:16:11.757236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:16:11.757236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:16:11.757236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:16:11.757236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:16:11.757236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:16:11.757236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:16:11.758267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:16:11.758267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:16:11.758267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:16:11.758267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:16:11.758267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:16:11.758267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:16:11.758267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:16:11.758267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:16:11.758267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:16:11.758267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:16:11.758267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:16:11.758267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:16:11.758267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 596/1188] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:16:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:16:11.763163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222c6b5-257c-437f-8540-4b250281c904 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:11.792386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:11.859120 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f222c6b5-257c-437f-8540-4b250281c904 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:11.891753 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff612a-cd4b-4954-937f-09374510dbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:11.892294 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff612a-cd4b-4954-937f-09374510dbef None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:11.892571 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff612a-cd4b-4954-937f-09374510dbef None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:11.892842 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ff612a-cd4b-4954-937f-09374510dbef None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:11.893493 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff612a-cd4b-4954-937f-09374510dbef None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:16:11.893792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff612a-cd4b-4954-937f-09374510dbef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:11.895167 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ff612a-cd4b-4954-937f-09374510dbef None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:11.902896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 593/1189] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:16:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:16:11.909303 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef910d9-3017-47bf-bd4a-0900de17526e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:12.005661 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef910d9-3017-47bf-bd4a-0900de17526e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:12.034424 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:12.039111 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e83bb-c4bf-4f4f-9cf1-a31b200f8fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:12.047127 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e83bb-c4bf-4f4f-9cf1-a31b200f8fcb None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:12.047127 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e83bb-c4bf-4f4f-9cf1-a31b200f8fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:12.047127 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e83bb-c4bf-4f4f-9cf1-a31b200f8fcb None admin] PATH_INFO: `/v3/users/eccec47b76704502ae0051f8fa121c0b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:12.047127 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e83bb-c4bf-4f4f-9cf1-a31b200f8fcb None admin] RBAC: Authorizing `identity:get_user(user_id=eccec47b76704502ae0051f8fa121c0b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:12.047127 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e83bb-c4bf-4f4f-9cf1-a31b200f8fcb None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:12.047127 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 597/1190] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:16:11 2022] GET /identity/v3/users/eccec47b76704502ae0051f8fa121c0b => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:16:12.049855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff612a-cd4b-4954-937f-09374510dbef None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:12.137807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:12.150296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ff612a-cd4b-4954-937f-09374510dbef None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:12.178120 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead9cbf-c9ba-45a8-bc8e-add040ddbf67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:12.178754 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead9cbf-c9ba-45a8-bc8e-add040ddbf67 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:12.179037 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead9cbf-c9ba-45a8-bc8e-add040ddbf67 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:12.179311 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead9cbf-c9ba-45a8-bc8e-add040ddbf67 None admin] PATH_INFO: `/v3/projects/5171161ba19a42c994dc9b82d6ac5537` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:12.180409 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead9cbf-c9ba-45a8-bc8e-add040ddbf67 None admin] RBAC: Authorizing `identity:get_project(project_id=5171161ba19a42c994dc9b82d6ac5537)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:12.181814 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead9cbf-c9ba-45a8-bc8e-add040ddbf67 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:12.183433 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 594/1191] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:16:12 2022] GET /identity/v3/projects/5171161ba19a42c994dc9b82d6ac5537 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:16:12.189744 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e83bb-c4bf-4f4f-9cf1-a31b200f8fcb None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:12.286603 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e83bb-c4bf-4f4f-9cf1-a31b200f8fcb None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:12.319407 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c067036-7147-4ddf-8900-dc7485fe702c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:12.320044 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c067036-7147-4ddf-8900-dc7485fe702c None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:12.320323 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c067036-7147-4ddf-8900-dc7485fe702c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:12.320596 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c067036-7147-4ddf-8900-dc7485fe702c None admin] PATH_INFO: `/v3/projects/5171161ba19a42c994dc9b82d6ac5537/users/eccec47b76704502ae0051f8fa121c0b/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:12.322455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c067036-7147-4ddf-8900-dc7485fe702c None admin] RBAC: Authorizing `identity:create_grant(project_id=5171161ba19a42c994dc9b82d6ac5537, user_id=eccec47b76704502ae0051f8fa121c0b, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:12.323908 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c067036-7147-4ddf-8900-dc7485fe702c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:12.404858 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 598/1192] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Sep 2 09:16:12 2022] PUT /identity/v3/projects/5171161ba19a42c994dc9b82d6ac5537/users/eccec47b76704502ae0051f8fa121c0b/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:16:14.262413 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8cbb3-ad47-4f54-82a4-7e353d6f6ac0 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:14.262979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8cbb3-ad47-4f54-82a4-7e353d6f6ac0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:14.263257 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8cbb3-ad47-4f54-82a4-7e353d6f6ac0 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:14.264604 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 595/1193] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:16:14.270375 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c24e1-9448-4106-8551-a948715b7354 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:14.270781 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c24e1-9448-4106-8551-a948715b7354 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:14.270963 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c24e1-9448-4106-8551-a948715b7354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:14.289029 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-743c24e1-9448-4106-8551-a948715b7354 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:16:14.290198 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-743c24e1-9448-4106-8551-a948715b7354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:16:14.345226 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:14.355793 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 599/1194] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:16:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:16:14.620437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8cbb3-ad47-4f54-82a4-7e353d6f6ac0 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:14.718380 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8cbb3-ad47-4f54-82a4-7e353d6f6ac0 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:14.751516 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a2552-8087-4fb2-9b09-2cfbbe17fabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:14.752415 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a2552-8087-4fb2-9b09-2cfbbe17fabf None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:14.752840 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a2552-8087-4fb2-9b09-2cfbbe17fabf None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:14.753162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a2552-8087-4fb2-9b09-2cfbbe17fabf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:14.764320 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a2552-8087-4fb2-9b09-2cfbbe17fabf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:16:14.769511 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-443a2552-8087-4fb2-9b09-2cfbbe17fabf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:16:14.769511 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:16:14.769511 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:16:14.769511 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:16:14.769511 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:16:14.769511 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:16:14.769511 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:16:14.769511 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:16:14.769511 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:16:14.769511 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:16:14.769511 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:16:14.769511 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:16:14.769511 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:16:14.769511 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:16:14.769511 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:16:14.769511 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:16:14.770655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:16:14.770655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:16:14.770655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:16:14.770655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:16:14.770655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:16:14.770655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:16:14.770655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:16:14.770655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:16:14.770655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:16:14.770655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:16:14.770655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:16:14.770655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:16:14.770655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:16:14.770655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:16:14.770655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:16:14.770655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:16:14.771618 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:16:14.771618 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:16:14.771618 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:16:14.771618 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:16:14.771618 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:16:14.771618 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:16:14.771618 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:16:14.771618 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:16:14.771618 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:16:14.771618 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:16:14.771618 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:16:14.771618 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:16:14.771618 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 596/1195] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:16:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:16:14.776518 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c24e1-9448-4106-8551-a948715b7354 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:14.871131 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c24e1-9448-4106-8551-a948715b7354 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:14.903552 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2bb1b-eab0-492b-af1b-597f49ad62a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:14.904153 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2bb1b-eab0-492b-af1b-597f49ad62a5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:14.904429 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2bb1b-eab0-492b-af1b-597f49ad62a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:14.904699 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2bb1b-eab0-492b-af1b-597f49ad62a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:14.905349 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f2bb1b-eab0-492b-af1b-597f49ad62a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:16:14.905650 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f2bb1b-eab0-492b-af1b-597f49ad62a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:14.907615 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f2bb1b-eab0-492b-af1b-597f49ad62a5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:14.915284 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 600/1196] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:16:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:16:14.921063 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a2552-8087-4fb2-9b09-2cfbbe17fabf None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:14.942728 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:15.016725 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a2552-8087-4fb2-9b09-2cfbbe17fabf None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:15.049065 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe8830b-68ed-455f-b43b-1173fd57fdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:15.049680 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8830b-68ed-455f-b43b-1173fd57fdb3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:15.049962 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8830b-68ed-455f-b43b-1173fd57fdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:15.050233 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8830b-68ed-455f-b43b-1173fd57fdb3 None admin] PATH_INFO: `/v3/users/eccec47b76704502ae0051f8fa121c0b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:15.051355 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe8830b-68ed-455f-b43b-1173fd57fdb3 None admin] RBAC: Authorizing `identity:get_user(user_id=eccec47b76704502ae0051f8fa121c0b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:15.052761 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe8830b-68ed-455f-b43b-1173fd57fdb3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:15.054303 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 597/1197] 10.222.0.57 () {56 vars in 1177 bytes} [Fri Sep 2 09:16:14 2022] GET /identity/v3/users/eccec47b76704502ae0051f8fa121c0b => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:16:15.060169 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2bb1b-eab0-492b-af1b-597f49ad62a5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:15.111166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:15.156891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2bb1b-eab0-492b-af1b-597f49ad62a5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:15.188607 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea089ff8-9c8d-4c59-83a5-9400c4ce9450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:15.189198 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea089ff8-9c8d-4c59-83a5-9400c4ce9450 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:15.189472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea089ff8-9c8d-4c59-83a5-9400c4ce9450 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:15.189739 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea089ff8-9c8d-4c59-83a5-9400c4ce9450 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:15.253825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea089ff8-9c8d-4c59-83a5-9400c4ce9450 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:15.255284 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea089ff8-9c8d-4c59-83a5-9400c4ce9450 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:15.303400 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:15.349868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 601/1198] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:16:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 09:16:15.355887 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe8830b-68ed-455f-b43b-1173fd57fdb3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:15.439861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:15.451092 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe8830b-68ed-455f-b43b-1173fd57fdb3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:15.483280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b4a93-88b0-4417-bad9-f50c054c5c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:15.483893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b4a93-88b0-4417-bad9-f50c054c5c71 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:15.484169 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b4a93-88b0-4417-bad9-f50c054c5c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:15.484932 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b4a93-88b0-4417-bad9-f50c054c5c71 None admin] PATH_INFO: `/v3/projects/5171161ba19a42c994dc9b82d6ac5537` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:15.486021 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b4a93-88b0-4417-bad9-f50c054c5c71 None admin] RBAC: Authorizing `identity:get_project(project_id=5171161ba19a42c994dc9b82d6ac5537)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:15.487448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b4a93-88b0-4417-bad9-f50c054c5c71 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:15.489017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 598/1199] 10.222.0.57 () {56 vars in 1186 bytes} [Fri Sep 2 09:16:15 2022] GET /identity/v3/projects/5171161ba19a42c994dc9b82d6ac5537 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:16:15.494988 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea089ff8-9c8d-4c59-83a5-9400c4ce9450 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:16:15.588781 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea089ff8-9c8d-4c59-83a5-9400c4ce9450 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:16:15.620847 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9537d08-43ab-4dbb-b36f-8d192a9f1fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:16:15.621427 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9537d08-43ab-4dbb-b36f-8d192a9f1fff None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:16:15.621701 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9537d08-43ab-4dbb-b36f-8d192a9f1fff None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:16:15.621966 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9537d08-43ab-4dbb-b36f-8d192a9f1fff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:16:15.622671 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9537d08-43ab-4dbb-b36f-8d192a9f1fff None admin] RBAC: Adding query filter params (role.id=f886b8e8889549e896a8fb3426cce9e1, scope.project.id=5171161ba19a42c994dc9b82d6ac5537, user.id=eccec47b76704502ae0051f8fa121c0b) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:16:15.623002 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9537d08-43ab-4dbb-b36f-8d192a9f1fff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:16:15.624405 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9537d08-43ab-4dbb-b36f-8d192a9f1fff None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:16:15.630487 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 602/1200] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:16:15 2022] GET /identity/v3/role_assignments?user.id=eccec47b76704502ae0051f8fa121c0b&scope.project.id=5171161ba19a42c994dc9b82d6ac5537&role.id=f886b8e8889549e896a8fb3426cce9e1 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:16:25.479385 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=22 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:16:25.628056 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:28.257147 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82946e56-cab5-4c1b-8128-bf60873ec638 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:28.257696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82946e56-cab5-4c1b-8128-bf60873ec638 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:28.257696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82946e56-cab5-4c1b-8128-bf60873ec638 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:28.259009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 599/1201] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:17:28.264233 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d8a80-4c64-4fe4-a819-dcc0dba4bd45 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:28.266882 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d8a80-4c64-4fe4-a819-dcc0dba4bd45 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:28.266882 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d8a80-4c64-4fe4-a819-dcc0dba4bd45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:28.282914 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-278d8a80-4c64-4fe4-a819-dcc0dba4bd45 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:28.284087 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-278d8a80-4c64-4fe4-a819-dcc0dba4bd45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:28.349361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 603/1202] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:17:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:17:28.617001 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82946e56-cab5-4c1b-8128-bf60873ec638 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:28.719234 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82946e56-cab5-4c1b-8128-bf60873ec638 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:28.752582 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66213925-fc8b-44bb-82fc-797ed59cedf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:28.753236 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66213925-fc8b-44bb-82fc-797ed59cedf1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:28.753505 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66213925-fc8b-44bb-82fc-797ed59cedf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:28.753773 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66213925-fc8b-44bb-82fc-797ed59cedf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:28.763394 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66213925-fc8b-44bb-82fc-797ed59cedf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:28.764860 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66213925-fc8b-44bb-82fc-797ed59cedf1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:28.778952 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-66213925-fc8b-44bb-82fc-797ed59cedf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:17:28.778952 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:28.778952 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:28.778952 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:28.778952 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:28.778952 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:28.778952 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:28.778952 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:28.778952 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:28.778952 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:28.778952 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:28.778952 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:28.778952 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:17:28.778952 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:17:28.778952 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:17:28.778952 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:17:28.780162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:28.780162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:28.780162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:28.780162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:28.780162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:28.780162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:28.780162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:28.780162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:28.780162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:28.780162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:28.780162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:28.780162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:28.780162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:28.780162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:28.780162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:28.780162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:28.780162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:17:28.781416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:17:28.781416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:17:28.781416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:17:28.781416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 600/1203] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:17:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:17:28.786541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d8a80-4c64-4fe4-a819-dcc0dba4bd45 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:28.884247 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d8a80-4c64-4fe4-a819-dcc0dba4bd45 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:28.912505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:28.918018 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634acd88-65a2-424b-82d5-e2e865d39cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:28.918618 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634acd88-65a2-424b-82d5-e2e865d39cde None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:28.918922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634acd88-65a2-424b-82d5-e2e865d39cde None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:28.919218 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634acd88-65a2-424b-82d5-e2e865d39cde None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:28.919846 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634acd88-65a2-424b-82d5-e2e865d39cde None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:28.920144 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634acd88-65a2-424b-82d5-e2e865d39cde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:28.921482 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634acd88-65a2-424b-82d5-e2e865d39cde None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:28.932401 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 604/1204] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:28.938673 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66213925-fc8b-44bb-82fc-797ed59cedf1 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:28.989787 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:29.035443 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66213925-fc8b-44bb-82fc-797ed59cedf1 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:29.067009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e560be6b-e6db-4c95-a83a-397ba60593cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:29.067583 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560be6b-e6db-4c95-a83a-397ba60593cb None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:29.067859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560be6b-e6db-4c95-a83a-397ba60593cb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:29.068135 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e560be6b-e6db-4c95-a83a-397ba60593cb None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:29.068776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e560be6b-e6db-4c95-a83a-397ba60593cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:29.070180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e560be6b-e6db-4c95-a83a-397ba60593cb None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:29.154218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 601/1205] 10.222.0.57 () {60 vars in 1132 bytes} [Fri Sep 2 09:17:28 2022] POST /identity/v3/users => generated 251 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:17:31.080996 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4a5ce-cd87-42fe-ba97-a12e0b881400 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:31.081532 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4a5ce-cd87-42fe-ba97-a12e0b881400 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:31.081532 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a4a5ce-cd87-42fe-ba97-a12e0b881400 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:31.082957 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 605/1206] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:17:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:17:31.088312 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7bca9-27e9-4bcb-967c-4030c65a841b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:31.088725 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7bca9-27e9-4bcb-967c-4030c65a841b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:31.088873 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7bca9-27e9-4bcb-967c-4030c65a841b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:31.107533 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-37d7bca9-27e9-4bcb-967c-4030c65a841b None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:31.108670 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-37d7bca9-27e9-4bcb-967c-4030c65a841b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:31.177790 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 602/1207] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:17:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:17:31.444691 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a4a5ce-cd87-42fe-ba97-a12e0b881400 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:31.544119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a4a5ce-cd87-42fe-ba97-a12e0b881400 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:31.577543 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92501d2c-e288-427f-9c4c-d9417370cd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:31.578159 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92501d2c-e288-427f-9c4c-d9417370cd59 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:31.578438 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92501d2c-e288-427f-9c4c-d9417370cd59 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:31.578728 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92501d2c-e288-427f-9c4c-d9417370cd59 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:31.588953 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92501d2c-e288-427f-9c4c-d9417370cd59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:17:31.593606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-92501d2c-e288-427f-9c4c-d9417370cd59 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:17:31.593606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:31.593606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:31.593606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:31.593606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:31.593606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:31.593606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:31.593606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:31.593606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:31.593606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:31.593606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:31.593606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:31.593606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:17:31.593606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:17:31.593606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:17:31.593606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:17:31.594759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:17:31.594759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:17:31.594759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:17:31.594759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:17:31.594759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:31.594759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:31.594759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:31.594759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:31.594759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:31.594759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:31.594759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:31.594759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:31.594759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:31.594759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:31.594759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:31.594759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:31.596322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:31.596322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:31.596322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:31.596322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:31.596322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:17:31.596322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:17:31.596322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:17:31.596322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:17:31.596322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:17:31.596322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:17:31.596322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:17:31.596322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:17:31.596322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 606/1208] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Sep 2 09:17:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:17:31.601361 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7bca9-27e9-4bcb-967c-4030c65a841b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:31.639947 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:31.697819 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7bca9-27e9-4bcb-967c-4030c65a841b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:31.730431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e2965-4261-4e0f-8428-b63f0b5b565f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:31.731063 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e2965-4261-4e0f-8428-b63f0b5b565f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:31.731338 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e2965-4261-4e0f-8428-b63f0b5b565f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:31.731603 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e2965-4261-4e0f-8428-b63f0b5b565f None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:31.732253 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e2965-4261-4e0f-8428-b63f0b5b565f None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:31.732547 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e2965-4261-4e0f-8428-b63f0b5b565f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:31.733894 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e2965-4261-4e0f-8428-b63f0b5b565f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:31.742299 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 603/1209] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Sep 2 09:17:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:31.748171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92501d2c-e288-427f-9c4c-d9417370cd59 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:31.800428 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:31.846050 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92501d2c-e288-427f-9c4c-d9417370cd59 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:31.879554 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd39fd4-2914-4f46-b08b-79fd38c5471f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:31.880759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd39fd4-2914-4f46-b08b-79fd38c5471f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:31.880759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd39fd4-2914-4f46-b08b-79fd38c5471f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:31.880759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd39fd4-2914-4f46-b08b-79fd38c5471f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:31.890924 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd39fd4-2914-4f46-b08b-79fd38c5471f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:31.892351 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd39fd4-2914-4f46-b08b-79fd38c5471f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:31.906575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-1bd39fd4-2914-4f46-b08b-79fd38c5471f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:17:31.906575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:31.906575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:31.906575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:31.906575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:31.906575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:31.906575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:31.906575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:31.906575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:31.906575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:31.906575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:31.906575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:31.906575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:17:31.906575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:17:31.906575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:17:31.906575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:17:31.920925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:31.920925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:31.920925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:31.920925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:31.920925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:31.920925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:31.920925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:31.920925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:31.920925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:31.920925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:31.920925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:31.920925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:31.920925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:31.920925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:31.920925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:31.920925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:31.920925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:17:31.922047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:17:31.922047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:17:31.922047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:17:31.922047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 607/1210] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:17:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:17:31.922325 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e2965-4261-4e0f-8428-b63f0b5b565f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:32.010493 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:32.013157 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e2965-4261-4e0f-8428-b63f0b5b565f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:32.046079 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12bd7f-52b7-42a4-9afc-c75172f93b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:32.046677 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12bd7f-52b7-42a4-9afc-c75172f93b3f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:32.046989 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12bd7f-52b7-42a4-9afc-c75172f93b3f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:32.047288 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12bd7f-52b7-42a4-9afc-c75172f93b3f None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:32.047893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a12bd7f-52b7-42a4-9afc-c75172f93b3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:32.048193 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a12bd7f-52b7-42a4-9afc-c75172f93b3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:32.049539 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a12bd7f-52b7-42a4-9afc-c75172f93b3f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:32.060903 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 604/1211] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:17:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:32.066888 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd39fd4-2914-4f46-b08b-79fd38c5471f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:32.165564 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd39fd4-2914-4f46-b08b-79fd38c5471f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:32.167110 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:32.198222 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbddd99-a362-47f9-8c8b-94d02792b87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:32.198872 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbddd99-a362-47f9-8c8b-94d02792b87f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:32.203369 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbddd99-a362-47f9-8c8b-94d02792b87f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:32.203369 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbddd99-a362-47f9-8c8b-94d02792b87f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:32.206903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbddd99-a362-47f9-8c8b-94d02792b87f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:32.208306 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbddd99-a362-47f9-8c8b-94d02792b87f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:32.219084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-5dbddd99-a362-47f9-8c8b-94d02792b87f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 02 09:17:32.219084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:32.219084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:32.219084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:32.219084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:32.219084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:32.219084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:32.219084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:32.219084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:32.219084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:32.219084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:32.219084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:32.219084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:17:32.219084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:17:32.219084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:17:32.219084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:17:32.220214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:32.220214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:32.220214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:17:32.220214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:17:32.220214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:17:32.220214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:17:32.220214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:32.220214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:32.220214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:32.220214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:32.220214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:32.220214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:32.220214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:32.220214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:32.220214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:32.220214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:32.220214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:32.220214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:32.221508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:32.221508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:32.221508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:17:32.221508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:17:32.221508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:17:32.221508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:17:32.221508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:17:32.221508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:17:32.221508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 02 09:17:32.221508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:17:32.221508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 608/1212] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:17:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:17:32.226215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12bd7f-52b7-42a4-9afc-c75172f93b3f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:32.321623 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12bd7f-52b7-42a4-9afc-c75172f93b3f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:32.353936 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47bbde-558f-4e77-8fbd-a79e268eb646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:32.354531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47bbde-558f-4e77-8fbd-a79e268eb646 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:32.354836 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47bbde-558f-4e77-8fbd-a79e268eb646 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:32.355111 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47bbde-558f-4e77-8fbd-a79e268eb646 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:32.355827 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce47bbde-558f-4e77-8fbd-a79e268eb646 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:32.356133 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce47bbde-558f-4e77-8fbd-a79e268eb646 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:32.357528 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce47bbde-558f-4e77-8fbd-a79e268eb646 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:32.375464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 605/1213] 10.222.0.57 () {56 vars in 1143 bytes} [Fri Sep 2 09:17:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:32.381732 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbddd99-a362-47f9-8c8b-94d02792b87f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:32.480216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbddd99-a362-47f9-8c8b-94d02792b87f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:32.512954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd57b4-37b5-4e86-826b-18743d4c5bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:32.513547 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd57b4-37b5-4e86-826b-18743d4c5bad None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:32.513827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd57b4-37b5-4e86-826b-18743d4c5bad None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:32.514098 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd57b4-37b5-4e86-826b-18743d4c5bad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:32.542997 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:32.580145 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cd57b4-37b5-4e86-826b-18743d4c5bad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:32.581534 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cd57b4-37b5-4e86-826b-18743d4c5bad None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:32.676924 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 609/1214] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:17:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 09:17:32.682858 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47bbde-558f-4e77-8fbd-a79e268eb646 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:32.702199 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:32.781112 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47bbde-558f-4e77-8fbd-a79e268eb646 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:32.814176 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffea14b-7ba0-47d5-bb1a-d3dd809486c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:32.815389 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffea14b-7ba0-47d5-bb1a-d3dd809486c1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:32.815674 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffea14b-7ba0-47d5-bb1a-d3dd809486c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:32.815948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffea14b-7ba0-47d5-bb1a-d3dd809486c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:32.827207 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffea14b-7ba0-47d5-bb1a-d3dd809486c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:32.828639 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffea14b-7ba0-47d5-bb1a-d3dd809486c1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:32.842995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-5ffea14b-7ba0-47d5-bb1a-d3dd809486c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:17:32.842995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:32.842995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:32.842995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:32.842995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:32.842995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:32.842995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:32.842995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:32.842995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:32.842995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:32.842995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:32.842995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:32.842995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:17:32.842995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:17:32.842995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:17:32.842995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:17:32.844180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:32.844180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:32.844180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:32.844180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:32.844180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:32.844180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:32.844180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:32.844180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:32.844180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:32.844180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:32.844180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:32.844180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:32.844180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:32.844180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:32.844180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:32.844180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:32.844180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:17:32.845815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:17:32.845815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:17:32.845815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:17:32.845815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 606/1215] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:17:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:17:32.849820 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd57b4-37b5-4e86-826b-18743d4c5bad None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:32.913531 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:32.950729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd57b4-37b5-4e86-826b-18743d4c5bad None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:32.983341 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5f2ab-899f-443e-937c-a8f57228b0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:32.983941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5f2ab-899f-443e-937c-a8f57228b0c5 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:32.984217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5f2ab-899f-443e-937c-a8f57228b0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:32.984482 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f5f2ab-899f-443e-937c-a8f57228b0c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:32.985112 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5f2ab-899f-443e-937c-a8f57228b0c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:32.985409 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5f2ab-899f-443e-937c-a8f57228b0c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:32.987389 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f5f2ab-899f-443e-937c-a8f57228b0c5 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:32.998225 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 610/1216] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:33.003890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffea14b-7ba0-47d5-bb1a-d3dd809486c1 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:33.072429 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:33.102266 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffea14b-7ba0-47d5-bb1a-d3dd809486c1 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:33.134838 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72e97bb-83dc-4ffa-a8e3-bfd94f754e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:33.135494 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e97bb-83dc-4ffa-a8e3-bfd94f754e2c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:33.135771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e97bb-83dc-4ffa-a8e3-bfd94f754e2c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:33.136040 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72e97bb-83dc-4ffa-a8e3-bfd94f754e2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:33.142434 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72e97bb-83dc-4ffa-a8e3-bfd94f754e2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:33.143916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72e97bb-83dc-4ffa-a8e3-bfd94f754e2c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:33.154907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d72e97bb-83dc-4ffa-a8e3-bfd94f754e2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:17:33.154907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:33.154907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:33.154907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:33.154907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:33.154907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:33.154907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:33.154907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:33.154907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:33.154907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:33.154907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:33.154907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:33.154907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:17:33.154907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:17:33.154907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:17:33.156025 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:17:33.156025 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:33.156025 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:33.156025 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:33.156025 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:33.156025 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:33.156025 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:33.156025 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:33.156025 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:33.156025 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:33.156025 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:33.156025 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:33.156025 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:33.156025 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:33.156025 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:33.156025 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:33.156025 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:33.156967 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:17:33.156967 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:17:33.156967 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:17:33.156967 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:17:33.156967 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:17:33.156967 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:17:33.156967 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:17:33.156967 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:17:33.156967 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 607/1217] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:17:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:17:33.162510 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5f2ab-899f-443e-937c-a8f57228b0c5 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:33.261484 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f5f2ab-899f-443e-937c-a8f57228b0c5 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:33.284433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:33.295638 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7664b91c-62ef-4e74-b598-a0951b1d27e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:33.296245 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664b91c-62ef-4e74-b598-a0951b1d27e9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:33.296525 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664b91c-62ef-4e74-b598-a0951b1d27e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:33.296791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664b91c-62ef-4e74-b598-a0951b1d27e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:33.297468 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7664b91c-62ef-4e74-b598-a0951b1d27e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:33.297769 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7664b91c-62ef-4e74-b598-a0951b1d27e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:33.299224 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7664b91c-62ef-4e74-b598-a0951b1d27e9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:33.311121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 611/1218] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:17:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:33.317516 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72e97bb-83dc-4ffa-a8e3-bfd94f754e2c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:33.416640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72e97bb-83dc-4ffa-a8e3-bfd94f754e2c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:33.443707 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:33.449438 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c394c2-725c-4f6f-a7f2-963416921639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:33.450033 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c394c2-725c-4f6f-a7f2-963416921639 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:33.450317 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c394c2-725c-4f6f-a7f2-963416921639 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:33.450591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c394c2-725c-4f6f-a7f2-963416921639 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:33.451344 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c394c2-725c-4f6f-a7f2-963416921639 None admin] RBAC: Adding query filter params (role.id=faf1138f11d34abe8a862f1a1f7c140a, scope.project.id=d18a5117a03b4f21ab3290a15680de66, user.id=5177fa4d168c4c219e09c7178bbbc790) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:33.451653 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c394c2-725c-4f6f-a7f2-963416921639 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:33.453078 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c394c2-725c-4f6f-a7f2-963416921639 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:33.458921 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 608/1219] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:17:33 2022] GET /identity/v3/role_assignments?user.id=5177fa4d168c4c219e09c7178bbbc790&scope.project.id=d18a5117a03b4f21ab3290a15680de66&role.id=faf1138f11d34abe8a862f1a1f7c140a => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:35.319051 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6fa2e-e567-402a-9931-1160ee579c3c None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:35.319575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6fa2e-e567-402a-9931-1160ee579c3c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:35.319575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6fa2e-e567-402a-9931-1160ee579c3c None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:35.320886 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 612/1220] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:17:35.325905 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b15662-455f-465e-8f9c-5fea016c1c0f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:35.326319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b15662-455f-465e-8f9c-5fea016c1c0f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:35.326454 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b15662-455f-465e-8f9c-5fea016c1c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:35.345055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f5b15662-455f-465e-8f9c-5fea016c1c0f None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:35.346226 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f5b15662-455f-465e-8f9c-5fea016c1c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:35.410326 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 609/1221] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:17:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:17:35.684206 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6fa2e-e567-402a-9931-1160ee579c3c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:35.785193 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6fa2e-e567-402a-9931-1160ee579c3c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:35.821434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865691e-68be-4f3a-b5d4-ad19ebaa369b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:35.822049 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865691e-68be-4f3a-b5d4-ad19ebaa369b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:35.822324 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865691e-68be-4f3a-b5d4-ad19ebaa369b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:35.822589 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865691e-68be-4f3a-b5d4-ad19ebaa369b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:35.832379 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d865691e-68be-4f3a-b5d4-ad19ebaa369b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:17:35.837054 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-d865691e-68be-4f3a-b5d4-ad19ebaa369b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:17:35.837054 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:35.837054 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:35.837054 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:35.837054 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:35.837054 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:35.837054 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:35.837054 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:35.837054 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:35.837054 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:35.837054 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:35.837054 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:35.837054 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:17:35.837054 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:17:35.837054 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:17:35.837054 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:17:35.842405 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:17:35.842405 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:17:35.842405 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:17:35.842405 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:17:35.842405 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:35.842405 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:35.842405 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:35.842405 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:35.842405 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:35.842405 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:35.842405 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:35.842405 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:35.842405 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:35.842405 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:35.842405 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:35.842405 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:35.843290 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:35.843290 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:35.843290 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:35.843290 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:35.843290 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:17:35.843290 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:17:35.843290 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:17:35.843290 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:17:35.843290 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:17:35.843290 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:17:35.843290 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:17:35.843290 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:17:35.843290 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 613/1222] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Sep 2 09:17:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:17:35.844485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b15662-455f-465e-8f9c-5fea016c1c0f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:35.943201 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b15662-455f-465e-8f9c-5fea016c1c0f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:35.965888 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:35.977029 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cd1d8-06ad-4be1-be96-9a79b2a4c781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:35.977680 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cd1d8-06ad-4be1-be96-9a79b2a4c781 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:35.981563 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cd1d8-06ad-4be1-be96-9a79b2a4c781 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:35.981563 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cd1d8-06ad-4be1-be96-9a79b2a4c781 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:35.981563 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6cd1d8-06ad-4be1-be96-9a79b2a4c781 None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:35.981563 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6cd1d8-06ad-4be1-be96-9a79b2a4c781 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:35.981563 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6cd1d8-06ad-4be1-be96-9a79b2a4c781 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:35.990538 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 610/1223] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Sep 2 09:17:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:35.996556 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865691e-68be-4f3a-b5d4-ad19ebaa369b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:36.040071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:36.094238 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865691e-68be-4f3a-b5d4-ad19ebaa369b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:36.129523 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4f087-29fb-4687-be92-9f55e9ad4d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:36.130211 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4f087-29fb-4687-be92-9f55e9ad4d6e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:36.130502 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4f087-29fb-4687-be92-9f55e9ad4d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:36.130806 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4f087-29fb-4687-be92-9f55e9ad4d6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:36.140583 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a4f087-29fb-4687-be92-9f55e9ad4d6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:36.142055 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a4f087-29fb-4687-be92-9f55e9ad4d6e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:36.156931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-74a4f087-29fb-4687-be92-9f55e9ad4d6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:17:36.156931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:36.156931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:36.156931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:36.156931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:36.156931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:36.156931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:36.156931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:36.156931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:36.156931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:36.156931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:36.156931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:36.156931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:17:36.156931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:17:36.156931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:17:36.156931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:17:36.162045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:36.162045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:36.162045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:36.162045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:36.162045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:36.162045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:36.162045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:36.162045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:36.162045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:36.162045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:36.162045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:36.162045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:36.162045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:36.162045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:36.162045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:36.162045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:36.162045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:17:36.163037 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:17:36.163037 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:17:36.163037 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:17:36.163037 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 614/1224] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:17:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:17:36.164052 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cd1d8-06ad-4be1-be96-9a79b2a4c781 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:36.261710 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cd1d8-06ad-4be1-be96-9a79b2a4c781 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:36.294372 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf96fd-4108-4fa2-9452-50cf90e19eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:36.294995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf96fd-4108-4fa2-9452-50cf90e19eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:36.295314 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf96fd-4108-4fa2-9452-50cf90e19eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:36.295585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf96fd-4108-4fa2-9452-50cf90e19eb5 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:36.296212 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf96fd-4108-4fa2-9452-50cf90e19eb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:36.296512 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf96fd-4108-4fa2-9452-50cf90e19eb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:36.297852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf96fd-4108-4fa2-9452-50cf90e19eb5 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:36.309872 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 611/1225] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:17:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:36.316333 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4f087-29fb-4687-be92-9f55e9ad4d6e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:36.408320 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:36.414644 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4f087-29fb-4687-be92-9f55e9ad4d6e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:36.447673 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446e095-07e3-4be5-a42d-321d2c718c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:36.448298 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446e095-07e3-4be5-a42d-321d2c718c5a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:36.448739 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446e095-07e3-4be5-a42d-321d2c718c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:36.448847 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446e095-07e3-4be5-a42d-321d2c718c5a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:36.456094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446e095-07e3-4be5-a42d-321d2c718c5a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:36.457531 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446e095-07e3-4be5-a42d-321d2c718c5a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:36.468634 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-1446e095-07e3-4be5-a42d-321d2c718c5a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 02 09:17:36.468634 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:36.468634 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:36.468634 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:36.468634 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:36.468634 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:36.468634 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:36.468634 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:36.468634 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:36.468634 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:36.468634 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:36.468634 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:36.468634 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:17:36.468634 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:17:36.468634 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:17:36.468634 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:17:36.469817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:36.469817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:36.469817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:17:36.469817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:17:36.469817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:17:36.469817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:17:36.469817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:36.469817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:36.469817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:36.469817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:36.469817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:36.469817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:36.469817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:36.469817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:36.469817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:36.469817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:36.469817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:36.469817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:36.471088 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:36.471088 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:36.471088 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:17:36.471088 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:17:36.471088 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:17:36.471088 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:17:36.471088 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:17:36.471088 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:17:36.471088 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 02 09:17:36.471088 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:17:36.474660 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 615/1226] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:17:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:17:36.475745 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf96fd-4108-4fa2-9452-50cf90e19eb5 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:36.523642 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:36.574552 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf96fd-4108-4fa2-9452-50cf90e19eb5 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:36.606651 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b12ef-1f50-4971-b116-e1827415143d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:36.607289 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b12ef-1f50-4971-b116-e1827415143d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:36.607566 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b12ef-1f50-4971-b116-e1827415143d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:36.607835 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b12ef-1f50-4971-b116-e1827415143d None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:36.608680 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b12ef-1f50-4971-b116-e1827415143d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:36.608982 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b12ef-1f50-4971-b116-e1827415143d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:36.610396 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b12ef-1f50-4971-b116-e1827415143d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:36.628177 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 612/1227] 10.222.0.57 () {56 vars in 1143 bytes} [Fri Sep 2 09:17:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:36.633896 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446e095-07e3-4be5-a42d-321d2c718c5a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:36.731372 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446e095-07e3-4be5-a42d-321d2c718c5a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:36.763728 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15081ab-3e89-4944-92b7-099d64ed27fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:36.764350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15081ab-3e89-4944-92b7-099d64ed27fc None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:36.764625 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15081ab-3e89-4944-92b7-099d64ed27fc None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:36.764898 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15081ab-3e89-4944-92b7-099d64ed27fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:36.775500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15081ab-3e89-4944-92b7-099d64ed27fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:36.776883 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15081ab-3e89-4944-92b7-099d64ed27fc None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:36.791500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-d15081ab-3e89-4944-92b7-099d64ed27fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:17:36.791500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:36.791500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:36.791500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:36.791500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:36.791500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:36.791500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:36.791500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:36.791500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:36.791500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:36.791500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:36.791500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:36.791500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:17:36.791500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:17:36.791500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:17:36.791500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:17:36.792662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:36.792662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:36.792662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:36.792662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:36.792662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:36.792662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:36.792662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:36.792662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:36.792662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:36.792662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:36.792662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:36.792662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:36.792662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:36.792662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:36.792662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:36.792662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:36.792662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:17:36.793574 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:17:36.793574 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:17:36.793574 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:17:36.793574 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 616/1228] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:17:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:17:36.798561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b12ef-1f50-4971-b116-e1827415143d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:36.898943 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b12ef-1f50-4971-b116-e1827415143d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:36.901336 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:36.932129 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b546f2d-4bc0-4a25-8bc5-f174bfe18dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:36.932722 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b546f2d-4bc0-4a25-8bc5-f174bfe18dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:36.932996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b546f2d-4bc0-4a25-8bc5-f174bfe18dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:36.933265 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b546f2d-4bc0-4a25-8bc5-f174bfe18dd7 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:36.933893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b546f2d-4bc0-4a25-8bc5-f174bfe18dd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:36.934190 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b546f2d-4bc0-4a25-8bc5-f174bfe18dd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:36.935675 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b546f2d-4bc0-4a25-8bc5-f174bfe18dd7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:36.947137 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 613/1229] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:17:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:36.952991 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15081ab-3e89-4944-92b7-099d64ed27fc None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:36.962810 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:37.052930 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15081ab-3e89-4944-92b7-099d64ed27fc None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:37.086832 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6effee-3235-4bcf-9e6d-f5768089fb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:37.087460 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6effee-3235-4bcf-9e6d-f5768089fb86 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:37.087742 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6effee-3235-4bcf-9e6d-f5768089fb86 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:37.088014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6effee-3235-4bcf-9e6d-f5768089fb86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:37.094466 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6effee-3235-4bcf-9e6d-f5768089fb86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:37.096020 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6effee-3235-4bcf-9e6d-f5768089fb86 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:37.107148 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-1f6effee-3235-4bcf-9e6d-f5768089fb86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:17:37.107148 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:37.107148 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:37.107148 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:37.107148 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:37.107148 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:37.107148 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:37.107148 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:37.107148 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:37.107148 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:37.107148 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:37.107148 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:37.107148 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:17:37.107148 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:17:37.107148 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:17:37.108294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:17:37.108294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:37.108294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:37.108294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:37.108294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:37.108294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:37.108294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:37.108294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:37.108294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:37.108294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:37.108294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:37.108294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:37.108294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:37.108294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:37.108294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:37.108294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:37.108294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:37.109396 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:17:37.109396 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:17:37.109396 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:17:37.109396 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:17:37.109396 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:17:37.109396 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:17:37.109396 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:17:37.109396 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:17:37.109396 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 617/1230] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:17:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:17:37.114847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b546f2d-4bc0-4a25-8bc5-f174bfe18dd7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:37.211992 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b546f2d-4bc0-4a25-8bc5-f174bfe18dd7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:37.244368 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f442f8-6fa7-48b4-878b-058ad0ef4c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:37.244963 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f442f8-6fa7-48b4-878b-058ad0ef4c0b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:37.245258 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f442f8-6fa7-48b4-878b-058ad0ef4c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:37.245508 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f442f8-6fa7-48b4-878b-058ad0ef4c0b None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:37.246183 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f442f8-6fa7-48b4-878b-058ad0ef4c0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:37.246478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f442f8-6fa7-48b4-878b-058ad0ef4c0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:37.247923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f442f8-6fa7-48b4-878b-058ad0ef4c0b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:37.260085 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 614/1231] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:17:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:37.266412 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6effee-3235-4bcf-9e6d-f5768089fb86 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:37.333370 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:37.367001 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6effee-3235-4bcf-9e6d-f5768089fb86 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:37.399797 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ff69a-674b-4ee1-a5e4-5ff3af38668c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:37.400453 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ff69a-674b-4ee1-a5e4-5ff3af38668c None admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:37.400733 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ff69a-674b-4ee1-a5e4-5ff3af38668c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:37.401005 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ff69a-674b-4ee1-a5e4-5ff3af38668c None admin] PATH_INFO: `/v3/projects/d18a5117a03b4f21ab3290a15680de66/users/5177fa4d168c4c219e09c7178bbbc790/roles/faf1138f11d34abe8a862f1a1f7c140a` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:37.415217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ff69a-674b-4ee1-a5e4-5ff3af38668c None admin] RBAC: Authorizing `identity:create_grant(project_id=d18a5117a03b4f21ab3290a15680de66, user_id=5177fa4d168c4c219e09c7178bbbc790, role_id=faf1138f11d34abe8a862f1a1f7c140a)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:37.420770 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ff69a-674b-4ee1-a5e4-5ff3af38668c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:37.496536 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 618/1232] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Sep 2 09:17:37 2022] PUT /identity/v3/projects/d18a5117a03b4f21ab3290a15680de66/users/5177fa4d168c4c219e09c7178bbbc790/roles/faf1138f11d34abe8a862f1a1f7c140a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:17:39.364630 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a57f1e-bbb3-4630-938d-f0785e70156a None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:39.365142 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a57f1e-bbb3-4630-938d-f0785e70156a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:39.365234 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a57f1e-bbb3-4630-938d-f0785e70156a None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:39.366516 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 615/1233] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:17:39.371779 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc01cf-5fa3-4a45-8db7-ad6c1cc6b4c9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:39.372148 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc01cf-5fa3-4a45-8db7-ad6c1cc6b4c9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:39.372330 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc01cf-5fa3-4a45-8db7-ad6c1cc6b4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:39.390524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-42bc01cf-5fa3-4a45-8db7-ad6c1cc6b4c9 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:39.391743 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-42bc01cf-5fa3-4a45-8db7-ad6c1cc6b4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:39.461776 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 619/1234] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:17:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:17:39.741683 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a57f1e-bbb3-4630-938d-f0785e70156a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:39.766478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:39.840433 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a57f1e-bbb3-4630-938d-f0785e70156a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:39.874902 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e4225-c625-4f2e-bc99-e0813a6520d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:39.875540 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e4225-c625-4f2e-bc99-e0813a6520d1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:39.875855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e4225-c625-4f2e-bc99-e0813a6520d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:39.876128 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e4225-c625-4f2e-bc99-e0813a6520d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:39.886342 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51e4225-c625-4f2e-bc99-e0813a6520d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:17:39.891189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-a51e4225-c625-4f2e-bc99-e0813a6520d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:17:39.891189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:39.891189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:39.891189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:39.891189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:39.891189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:39.891189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:39.891189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:39.891189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:39.891189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:39.891189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:39.891189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:39.891189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:17:39.891189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:17:39.891189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:17:39.891189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:17:39.892384 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:17:39.892384 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:17:39.892384 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:17:39.892384 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:17:39.892384 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:39.892384 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:39.892384 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:39.892384 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:39.892384 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:39.892384 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:39.892384 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:39.892384 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:39.892384 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:39.892384 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:39.892384 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:39.892384 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:39.893425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:39.893425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:39.893425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:39.893425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:39.893425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:17:39.893425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:17:39.893425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:17:39.893425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:17:39.893425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:17:39.893425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:17:39.893425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 02 09:17:39.893425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:17:39.893425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 616/1235] 10.222.0.57 () {56 vars in 1102 bytes} [Fri Sep 2 09:17:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:17:39.898399 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc01cf-5fa3-4a45-8db7-ad6c1cc6b4c9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:39.944495 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:39.996519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc01cf-5fa3-4a45-8db7-ad6c1cc6b4c9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:40.030027 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1345ae-3e2f-4809-9815-3d6f926a5f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:40.030623 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1345ae-3e2f-4809-9815-3d6f926a5f91 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:40.030928 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1345ae-3e2f-4809-9815-3d6f926a5f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:40.031197 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1345ae-3e2f-4809-9815-3d6f926a5f91 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:40.031859 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1345ae-3e2f-4809-9815-3d6f926a5f91 None admin] RBAC: Adding query filter params (name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:40.032163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1345ae-3e2f-4809-9815-3d6f926a5f91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:40.034073 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1345ae-3e2f-4809-9815-3d6f926a5f91 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:40.041586 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 620/1236] 10.222.0.57 () {56 vars in 1103 bytes} [Fri Sep 2 09:17:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:40.047443 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e4225-c625-4f2e-bc99-e0813a6520d1 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:40.130331 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:40.146264 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e4225-c625-4f2e-bc99-e0813a6520d1 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:40.178217 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8809ef-4cf0-4889-88ed-8b9df5d3525e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:40.178875 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8809ef-4cf0-4889-88ed-8b9df5d3525e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:40.179153 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8809ef-4cf0-4889-88ed-8b9df5d3525e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:40.179419 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8809ef-4cf0-4889-88ed-8b9df5d3525e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:40.190384 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8809ef-4cf0-4889-88ed-8b9df5d3525e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:40.196114 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8809ef-4cf0-4889-88ed-8b9df5d3525e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:40.205799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-cb8809ef-4cf0-4889-88ed-8b9df5d3525e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:17:40.205799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:40.205799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:40.205799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:40.205799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:40.205799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:40.205799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:40.205799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:40.205799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:40.205799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:40.205799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:40.205799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:40.205799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:17:40.205799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:17:40.205799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:17:40.205799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:17:40.206948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:40.206948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:40.206948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:40.206948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:40.206948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:40.206948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:40.206948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:40.206948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:40.206948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:40.206948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:40.206948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:40.206948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:40.206948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:40.206948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:40.206948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:40.206948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:40.206948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:17:40.207884 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:17:40.207884 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:17:40.207884 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:17:40.207884 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 617/1237] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:17:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:17:40.212719 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1345ae-3e2f-4809-9815-3d6f926a5f91 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:40.309645 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1345ae-3e2f-4809-9815-3d6f926a5f91 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:40.318230 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:40.341882 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cd20d3-405c-432f-a92e-efe22cb4c808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:40.342479 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd20d3-405c-432f-a92e-efe22cb4c808 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:40.342783 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd20d3-405c-432f-a92e-efe22cb4c808 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:40.343056 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cd20d3-405c-432f-a92e-efe22cb4c808 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:40.343686 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cd20d3-405c-432f-a92e-efe22cb4c808 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:40.343986 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cd20d3-405c-432f-a92e-efe22cb4c808 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:40.345326 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cd20d3-405c-432f-a92e-efe22cb4c808 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:40.357237 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 621/1238] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:17:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:40.363095 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8809ef-4cf0-4889-88ed-8b9df5d3525e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:40.461548 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8809ef-4cf0-4889-88ed-8b9df5d3525e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:40.494677 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32905d4-ea93-43b9-bbc1-474f460dff7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:40.495324 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32905d4-ea93-43b9-bbc1-474f460dff7a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:40.495604 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32905d4-ea93-43b9-bbc1-474f460dff7a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:40.495875 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32905d4-ea93-43b9-bbc1-474f460dff7a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:40.498451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:40.503744 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32905d4-ea93-43b9-bbc1-474f460dff7a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:40.505576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32905d4-ea93-43b9-bbc1-474f460dff7a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:40.516098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-e32905d4-ea93-43b9-bbc1-474f460dff7a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 02 09:17:40.516098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:40.516098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:40.516098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:40.516098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:40.516098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:40.516098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:40.516098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:40.516098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:40.516098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:40.516098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:40.516098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:40.516098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:17:40.516098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:17:40.516098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:17:40.516098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:17:40.517387 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:40.517387 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:40.517387 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:17:40.517387 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:17:40.517387 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:17:40.517387 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:17:40.517387 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:40.517387 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:40.517387 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:40.517387 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:40.517387 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:40.517387 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:40.517387 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:40.517387 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:40.517387 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:40.517387 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:40.517387 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:40.517387 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:40.519163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:40.519163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:40.519163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:17:40.519163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:17:40.519163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:17:40.519163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:17:40.519163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:17:40.519163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:17:40.519163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 02 09:17:40.519163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:17:40.519163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 618/1239] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:17:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:17:40.523642 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cd20d3-405c-432f-a92e-efe22cb4c808 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:40.622092 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cd20d3-405c-432f-a92e-efe22cb4c808 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:40.656231 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb51b1-5fa4-4af4-ac80-3a9a58efb401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:40.656844 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb51b1-5fa4-4af4-ac80-3a9a58efb401 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:40.657122 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb51b1-5fa4-4af4-ac80-3a9a58efb401 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:40.657391 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb51b1-5fa4-4af4-ac80-3a9a58efb401 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:40.658142 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb51b1-5fa4-4af4-ac80-3a9a58efb401 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:40.658447 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb51b1-5fa4-4af4-ac80-3a9a58efb401 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:40.659936 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb51b1-5fa4-4af4-ac80-3a9a58efb401 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:40.677407 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 622/1240] 10.222.0.57 () {56 vars in 1143 bytes} [Fri Sep 2 09:17:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:40.683332 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32905d4-ea93-43b9-bbc1-474f460dff7a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:40.780078 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32905d4-ea93-43b9-bbc1-474f460dff7a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:40.812717 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a1101-1ee4-4d3a-9edf-90e7215b335e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:40.813315 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a1101-1ee4-4d3a-9edf-90e7215b335e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:40.813595 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a1101-1ee4-4d3a-9edf-90e7215b335e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:40.813864 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a1101-1ee4-4d3a-9edf-90e7215b335e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:40.872847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:40.879868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a1101-1ee4-4d3a-9edf-90e7215b335e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:40.881309 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a1101-1ee4-4d3a-9edf-90e7215b335e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:40.984126 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 619/1241] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:17:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 09:17:40.990565 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb51b1-5fa4-4af4-ac80-3a9a58efb401 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:41.017939 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:41.089922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb51b1-5fa4-4af4-ac80-3a9a58efb401 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:41.124281 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26ce93-b0a9-40e6-a9ea-2cf65f0179e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:41.124846 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26ce93-b0a9-40e6-a9ea-2cf65f0179e1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:41.125130 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26ce93-b0a9-40e6-a9ea-2cf65f0179e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:41.125405 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26ce93-b0a9-40e6-a9ea-2cf65f0179e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:41.136062 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26ce93-b0a9-40e6-a9ea-2cf65f0179e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:41.137519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26ce93-b0a9-40e6-a9ea-2cf65f0179e1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:41.151885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ec26ce93-b0a9-40e6-a9ea-2cf65f0179e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:17:41.151885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:41.151885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:41.151885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:41.151885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:41.151885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:41.151885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:41.151885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:41.151885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:41.151885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:41.151885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:41.151885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:41.151885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:17:41.151885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:17:41.151885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:17:41.151885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:17:41.157620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:41.157620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:41.157620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:41.157620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:41.157620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:41.157620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:41.157620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:41.157620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:41.157620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:41.157620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:41.157620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:41.157620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:41.157620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:41.157620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:41.157620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:41.157620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:41.157620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:17:41.159390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:17:41.159390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:17:41.159390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:17:41.159390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 623/1242] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:17:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:17:41.159693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a1101-1ee4-4d3a-9edf-90e7215b335e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:41.250217 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:41.255704 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a1101-1ee4-4d3a-9edf-90e7215b335e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:41.288334 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f823ae-a63e-42b1-b463-3c88a0ed037f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:41.288928 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f823ae-a63e-42b1-b463-3c88a0ed037f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:41.289205 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f823ae-a63e-42b1-b463-3c88a0ed037f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:41.289567 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f823ae-a63e-42b1-b463-3c88a0ed037f None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:41.290191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f823ae-a63e-42b1-b463-3c88a0ed037f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:41.290488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f823ae-a63e-42b1-b463-3c88a0ed037f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:41.291864 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f823ae-a63e-42b1-b463-3c88a0ed037f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:41.303640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 620/1243] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:17:41.312563 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26ce93-b0a9-40e6-a9ea-2cf65f0179e1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:41.386177 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:41.406401 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26ce93-b0a9-40e6-a9ea-2cf65f0179e1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:41.439306 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b35f38f-d984-48e3-98c1-0df1f3eadd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:41.439926 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35f38f-d984-48e3-98c1-0df1f3eadd2c None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:41.440205 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35f38f-d984-48e3-98c1-0df1f3eadd2c None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:41.440479 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b35f38f-d984-48e3-98c1-0df1f3eadd2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:41.447021 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b35f38f-d984-48e3-98c1-0df1f3eadd2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:41.448445 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b35f38f-d984-48e3-98c1-0df1f3eadd2c None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:41.459788 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-0b35f38f-d984-48e3-98c1-0df1f3eadd2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:17:41.459788 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:41.459788 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:41.459788 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:41.459788 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:41.459788 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:41.459788 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:41.459788 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:41.459788 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:41.459788 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:41.459788 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:41.459788 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:41.459788 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:17:41.459788 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:17:41.459788 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:17:41.460891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:17:41.460891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:41.460891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:41.460891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:41.460891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:41.460891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:41.460891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:41.460891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:41.460891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:41.460891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:41.460891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:41.460891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:41.460891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:41.460891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:41.460891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:41.460891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:41.460891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:41.462095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:17:41.462095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:17:41.462095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:17:41.462095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:17:41.462095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:17:41.462095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:17:41.462095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:17:41.462095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:17:41.462095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 624/1244] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:17:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:17:41.468728 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f823ae-a63e-42b1-b463-3c88a0ed037f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:41.569871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f823ae-a63e-42b1-b463-3c88a0ed037f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:41.603238 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05dcdb-4db6-4153-bf29-4901ffd57af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:41.603877 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05dcdb-4db6-4153-bf29-4901ffd57af5 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:41.604175 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05dcdb-4db6-4153-bf29-4901ffd57af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:41.604441 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05dcdb-4db6-4153-bf29-4901ffd57af5 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:41.605118 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05dcdb-4db6-4153-bf29-4901ffd57af5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:41.605417 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05dcdb-4db6-4153-bf29-4901ffd57af5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:41.606845 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05dcdb-4db6-4153-bf29-4901ffd57af5 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:41.618637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 621/1245] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:17:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:41.624745 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b35f38f-d984-48e3-98c1-0df1f3eadd2c None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:41.723071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b35f38f-d984-48e3-98c1-0df1f3eadd2c None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:41.755520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae9270b-9b24-4f59-8212-b96526b0c026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:41.759989 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9270b-9b24-4f59-8212-b96526b0c026 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:41.759989 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9270b-9b24-4f59-8212-b96526b0c026 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:41.759989 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9270b-9b24-4f59-8212-b96526b0c026 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:41.759989 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae9270b-9b24-4f59-8212-b96526b0c026 None admin] RBAC: Adding query filter params (role.id=faf1138f11d34abe8a862f1a1f7c140a, scope.project.id=d18a5117a03b4f21ab3290a15680de66, user.id=5177fa4d168c4c219e09c7178bbbc790) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:41.759989 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae9270b-9b24-4f59-8212-b96526b0c026 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:41.759989 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae9270b-9b24-4f59-8212-b96526b0c026 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:41.764832 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 625/1246] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:17:41 2022] GET /identity/v3/role_assignments?user.id=5177fa4d168c4c219e09c7178bbbc790&scope.project.id=d18a5117a03b4f21ab3290a15680de66&role.id=faf1138f11d34abe8a862f1a1f7c140a => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:43.703829 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289db49d-6ad4-4e14-89b1-715f62bedc97 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:43.704400 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289db49d-6ad4-4e14-89b1-715f62bedc97 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:43.704400 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289db49d-6ad4-4e14-89b1-715f62bedc97 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:43.705741 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 622/1247] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:17:43.710801 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7672d35-4811-4879-89fa-d91f2c0762a8 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:43.711217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7672d35-4811-4879-89fa-d91f2c0762a8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:43.711361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7672d35-4811-4879-89fa-d91f2c0762a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:43.717623 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:43.730318 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a7672d35-4811-4879-89fa-d91f2c0762a8 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:43.731556 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a7672d35-4811-4879-89fa-d91f2c0762a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:43.798131 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 626/1248] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:17:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 09:17:44.066853 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289db49d-6ad4-4e14-89b1-715f62bedc97 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:44.085156 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:44.168583 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289db49d-6ad4-4e14-89b1-715f62bedc97 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:44.201861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f77ef8-6413-4aa3-a328-bc222b8cca05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:44.202489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f77ef8-6413-4aa3-a328-bc222b8cca05 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:44.202798 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f77ef8-6413-4aa3-a328-bc222b8cca05 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:44.203072 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f77ef8-6413-4aa3-a328-bc222b8cca05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:44.213608 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f77ef8-6413-4aa3-a328-bc222b8cca05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:17:44.218424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-e0f77ef8-6413-4aa3-a328-bc222b8cca05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:17:44.218424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:44.218424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:44.218424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:44.218424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:44.218424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:44.218424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:44.218424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:44.218424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:44.218424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:44.218424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:44.218424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:44.218424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:17:44.218424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:17:44.218424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:17:44.218424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:17:44.219602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:17:44.219602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:17:44.219602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:17:44.219602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:17:44.219602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:44.219602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:44.219602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:44.219602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:44.219602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:44.219602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:44.219602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:44.219602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:44.219602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:44.219602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:44.219602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:44.219602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:44.220445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:44.220445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:44.220445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:44.220445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:44.220445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:17:44.220445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:17:44.220445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:17:44.220445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:17:44.220445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:17:44.220445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:17:44.220445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:17:44.220445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:17:44.222853 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 623/1249] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:17:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:17:44.225532 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7672d35-4811-4879-89fa-d91f2c0762a8 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:44.323486 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7672d35-4811-4879-89fa-d91f2c0762a8 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:44.353338 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:44.358124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766ee06-5db0-4e76-9d5a-49f177c924fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:44.358757 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766ee06-5db0-4e76-9d5a-49f177c924fa None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:44.359044 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766ee06-5db0-4e76-9d5a-49f177c924fa None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:44.359318 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8766ee06-5db0-4e76-9d5a-49f177c924fa None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:44.359981 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8766ee06-5db0-4e76-9d5a-49f177c924fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:44.360286 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8766ee06-5db0-4e76-9d5a-49f177c924fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:44.361674 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8766ee06-5db0-4e76-9d5a-49f177c924fa None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:44.369467 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 627/1250] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:17:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:44.375243 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f77ef8-6413-4aa3-a328-bc222b8cca05 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:44.451099 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:44.473522 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f77ef8-6413-4aa3-a328-bc222b8cca05 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:44.507212 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbbcb0c-5325-49f4-9822-e8256ee36422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:44.507835 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbbcb0c-5325-49f4-9822-e8256ee36422 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:44.508114 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbbcb0c-5325-49f4-9822-e8256ee36422 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:44.508420 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbbcb0c-5325-49f4-9822-e8256ee36422 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:44.518768 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbbcb0c-5325-49f4-9822-e8256ee36422 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:44.520203 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbbcb0c-5325-49f4-9822-e8256ee36422 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:44.534541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-2fbbcb0c-5325-49f4-9822-e8256ee36422 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:17:44.534541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:44.534541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:44.534541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:44.534541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:44.534541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:44.534541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:44.534541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:44.534541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:44.534541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:44.534541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:44.534541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:44.534541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:17:44.534541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:17:44.534541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:17:44.534541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:17:44.536059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:44.536059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:44.536059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:44.536059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:44.536059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:44.536059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:44.536059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:44.536059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:44.536059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:44.536059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:44.536059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:44.536059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:44.536059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:44.536059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:44.536059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:44.536059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:44.536059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:17:44.536981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:17:44.536981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:17:44.536981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:17:44.536981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 624/1251] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:17:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:17:44.541689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766ee06-5db0-4e76-9d5a-49f177c924fa None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:44.640485 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8766ee06-5db0-4e76-9d5a-49f177c924fa None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:44.672809 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3317a20-b89b-492d-8053-eab388b8428e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:44.673921 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3317a20-b89b-492d-8053-eab388b8428e None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:44.674193 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3317a20-b89b-492d-8053-eab388b8428e None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:44.674457 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3317a20-b89b-492d-8053-eab388b8428e None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:44.675119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3317a20-b89b-492d-8053-eab388b8428e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:44.675415 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3317a20-b89b-492d-8053-eab388b8428e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:44.676746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3317a20-b89b-492d-8053-eab388b8428e None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:44.688309 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 628/1252] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:44.694791 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbbcb0c-5325-49f4-9822-e8256ee36422 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:44.790962 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbbcb0c-5325-49f4-9822-e8256ee36422 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:44.816540 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:44.824523 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304a914-15df-4f6b-863b-53e1ecba0583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:44.825157 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304a914-15df-4f6b-863b-53e1ecba0583 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:44.825479 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304a914-15df-4f6b-863b-53e1ecba0583 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:44.825758 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304a914-15df-4f6b-863b-53e1ecba0583 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:44.833290 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304a914-15df-4f6b-863b-53e1ecba0583 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:44.834697 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1304a914-15df-4f6b-863b-53e1ecba0583 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:44.845366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-1304a914-15df-4f6b-863b-53e1ecba0583 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 02 09:17:44.845366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:44.845366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:44.845366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:44.845366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:44.845366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:44.845366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:44.845366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:44.845366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:44.845366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:44.845366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:44.845366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:44.845366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:17:44.845366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:17:44.845366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:17:44.845366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:17:44.846472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:44.846472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:44.846472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:17:44.846472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:17:44.846472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:17:44.846472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:17:44.846472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:44.846472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:44.846472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:44.846472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:44.846472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:44.846472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:44.846472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:44.846472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:44.846472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:44.846472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:44.846472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:44.846472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:44.847686 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:44.847686 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:44.847686 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:17:44.847686 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:17:44.847686 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:17:44.847686 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:17:44.847686 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:17:44.847686 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:17:44.847686 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 02 09:17:44.847686 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:17:44.847686 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 625/1253] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:17:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:17:44.852341 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3317a20-b89b-492d-8053-eab388b8428e None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:44.907009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:44.950218 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3317a20-b89b-492d-8053-eab388b8428e None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:44.984925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b915cb-55a2-4cee-8dbe-1a58de7b23e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:44.985531 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b915cb-55a2-4cee-8dbe-1a58de7b23e9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:44.986381 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b915cb-55a2-4cee-8dbe-1a58de7b23e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:44.986698 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b915cb-55a2-4cee-8dbe-1a58de7b23e9 None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:44.987511 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b915cb-55a2-4cee-8dbe-1a58de7b23e9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:44.987821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b915cb-55a2-4cee-8dbe-1a58de7b23e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:44.989249 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b915cb-55a2-4cee-8dbe-1a58de7b23e9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:45.006000 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 629/1254] 10.222.0.57 () {56 vars in 1143 bytes} [Fri Sep 2 09:17:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:45.011762 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304a914-15df-4f6b-863b-53e1ecba0583 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:45.112235 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304a914-15df-4f6b-863b-53e1ecba0583 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:45.144643 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e863a7-1ecb-4232-ab5e-3a350ec32e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:45.145254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e863a7-1ecb-4232-ab5e-3a350ec32e32 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:45.145537 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e863a7-1ecb-4232-ab5e-3a350ec32e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:45.145808 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e863a7-1ecb-4232-ab5e-3a350ec32e32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:45.201191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:45.212946 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e863a7-1ecb-4232-ab5e-3a350ec32e32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:45.214377 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e863a7-1ecb-4232-ab5e-3a350ec32e32 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:45.309633 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 626/1255] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:17:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 02 09:17:45.315770 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b915cb-55a2-4cee-8dbe-1a58de7b23e9 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:45.414330 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b915cb-55a2-4cee-8dbe-1a58de7b23e9 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:45.427223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:45.447989 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d0f7f-dc1b-4cb6-91e3-1ee1d067b21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:45.448608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d0f7f-dc1b-4cb6-91e3-1ee1d067b21f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:45.448886 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d0f7f-dc1b-4cb6-91e3-1ee1d067b21f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:45.449157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d0f7f-dc1b-4cb6-91e3-1ee1d067b21f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:45.464445 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d0f7f-dc1b-4cb6-91e3-1ee1d067b21f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:45.464445 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d0f7f-dc1b-4cb6-91e3-1ee1d067b21f None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:45.475438 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-b54d0f7f-dc1b-4cb6-91e3-1ee1d067b21f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:17:45.475438 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:45.475438 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:45.475438 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:45.475438 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:45.475438 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:45.475438 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:45.475438 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:45.475438 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:45.475438 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:45.475438 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:45.475438 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:45.475438 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:17:45.475438 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:17:45.475438 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:17:45.475438 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:17:45.476343 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:45.476343 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:45.476343 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:45.476343 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:45.476343 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:45.476343 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:45.476343 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:45.476343 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:45.476343 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:45.476343 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:45.476343 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:45.476343 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:45.476343 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:45.476343 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:45.476343 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:45.476343 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:45.476343 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:17:45.477567 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:17:45.477567 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:17:45.477567 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:17:45.477567 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 630/1256] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:17:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:17:45.482206 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e863a7-1ecb-4232-ab5e-3a350ec32e32 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:45.566865 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:45.578240 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e863a7-1ecb-4232-ab5e-3a350ec32e32 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:45.611730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822efd81-ef2e-4a27-9121-7ad4478408e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:45.612332 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822efd81-ef2e-4a27-9121-7ad4478408e4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:45.612612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822efd81-ef2e-4a27-9121-7ad4478408e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:45.612889 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822efd81-ef2e-4a27-9121-7ad4478408e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:45.613525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822efd81-ef2e-4a27-9121-7ad4478408e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:45.613944 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822efd81-ef2e-4a27-9121-7ad4478408e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:45.615326 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822efd81-ef2e-4a27-9121-7ad4478408e4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:45.626148 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 627/1257] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:45.632387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d0f7f-dc1b-4cb6-91e3-1ee1d067b21f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:45.731172 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d0f7f-dc1b-4cb6-91e3-1ee1d067b21f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:45.764984 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9734a2-b6c4-484a-8a9c-071021073980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:45.765607 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9734a2-b6c4-484a-8a9c-071021073980 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:45.765885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9734a2-b6c4-484a-8a9c-071021073980 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:45.766155 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9734a2-b6c4-484a-8a9c-071021073980 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:45.773106 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9734a2-b6c4-484a-8a9c-071021073980 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:45.774518 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9734a2-b6c4-484a-8a9c-071021073980 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:45.786179 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-aa9734a2-b6c4-484a-8a9c-071021073980 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:17:45.786179 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:45.786179 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:45.786179 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:45.786179 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:45.786179 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:45.786179 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:45.786179 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:45.786179 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:45.786179 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:45.786179 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:45.786179 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:45.786179 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:17:45.786179 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:17:45.786179 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:17:45.790811 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:17:45.790811 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:45.790811 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:45.790811 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:45.790811 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:45.790811 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:45.790811 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:45.790811 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:45.790811 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:45.790811 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:45.790811 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:45.790811 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:45.790811 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:45.790811 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:45.790811 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:45.790811 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:45.790811 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:45.791961 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:17:45.791961 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:17:45.791961 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:17:45.791961 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:17:45.791961 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:17:45.791961 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:17:45.791961 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:17:45.791961 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:17:45.791961 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 631/1258] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:17:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:17:45.793378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822efd81-ef2e-4a27-9121-7ad4478408e4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:45.892752 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822efd81-ef2e-4a27-9121-7ad4478408e4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:45.925263 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66337f-039a-4e06-bf94-c49d2125d937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:45.925868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66337f-039a-4e06-bf94-c49d2125d937 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:45.926146 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66337f-039a-4e06-bf94-c49d2125d937 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:45.926473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66337f-039a-4e06-bf94-c49d2125d937 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:45.927123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe66337f-039a-4e06-bf94-c49d2125d937 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:45.927474 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe66337f-039a-4e06-bf94-c49d2125d937 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:45.928876 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe66337f-039a-4e06-bf94-c49d2125d937 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:45.941189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 628/1259] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:17:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:45.947465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9734a2-b6c4-484a-8a9c-071021073980 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:45.972643 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:46.045048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9734a2-b6c4-484a-8a9c-071021073980 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:46.078497 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f5392d-27fb-4955-9335-32abbcd16392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:46.079127 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5392d-27fb-4955-9335-32abbcd16392 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:46.079408 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5392d-27fb-4955-9335-32abbcd16392 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:46.080164 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5392d-27fb-4955-9335-32abbcd16392 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:46.080880 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f5392d-27fb-4955-9335-32abbcd16392 None admin] RBAC: Adding query filter params (role.id=f886b8e8889549e896a8fb3426cce9e1, scope.project.id=d18a5117a03b4f21ab3290a15680de66, user.id=5177fa4d168c4c219e09c7178bbbc790) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:46.081187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f5392d-27fb-4955-9335-32abbcd16392 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:46.082601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f5392d-27fb-4955-9335-32abbcd16392 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:46.088017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 632/1260] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:17:45 2022] GET /identity/v3/role_assignments?user.id=5177fa4d168c4c219e09c7178bbbc790&scope.project.id=d18a5117a03b4f21ab3290a15680de66&role.id=f886b8e8889549e896a8fb3426cce9e1 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:47.951597 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b4ac7-098f-431d-a0cc-9fca2bb6c87a None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:47.952114 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b4ac7-098f-431d-a0cc-9fca2bb6c87a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:47.952223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b4ac7-098f-431d-a0cc-9fca2bb6c87a None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:47.953608 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 629/1261] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:17:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:17:47.959236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed7638-6a3e-40ce-9d76-97cc86de0990 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:47.959656 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed7638-6a3e-40ce-9d76-97cc86de0990 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:47.959769 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed7638-6a3e-40ce-9d76-97cc86de0990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:47.977564 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e6ed7638-6a3e-40ce-9d76-97cc86de0990 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:47.978769 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e6ed7638-6a3e-40ce-9d76-97cc86de0990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:48.044132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 633/1262] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:17:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:17:48.312969 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b4ac7-098f-431d-a0cc-9fca2bb6c87a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:48.324445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:48.414253 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b4ac7-098f-431d-a0cc-9fca2bb6c87a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:48.448167 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0585f4-416c-4626-a64f-1b14d835e96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:48.448787 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0585f4-416c-4626-a64f-1b14d835e96e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:48.449067 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0585f4-416c-4626-a64f-1b14d835e96e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:48.449340 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0585f4-416c-4626-a64f-1b14d835e96e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:48.459276 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0585f4-416c-4626-a64f-1b14d835e96e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:17:48.464045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-ef0585f4-416c-4626-a64f-1b14d835e96e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:17:48.464045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:48.464045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:48.464045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:48.464045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:48.464045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:48.464045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:48.464045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:48.464045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:48.464045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:48.464045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:48.464045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:48.464045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:17:48.464045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:17:48.464045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:17:48.464045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:17:48.465283 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:17:48.465283 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:17:48.465283 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:17:48.465283 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:17:48.465283 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:48.465283 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:48.465283 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:48.465283 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:48.465283 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:48.465283 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:48.465283 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:48.465283 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:48.465283 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:48.465283 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:48.465283 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:48.465283 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:48.466116 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:48.466116 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:48.466116 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:48.466116 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:48.466116 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:17:48.466116 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:17:48.466116 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:17:48.466116 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:17:48.466116 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:17:48.466116 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:17:48.466116 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:17:48.466116 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:17:48.466116 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 630/1263] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:17:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:17:48.471216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed7638-6a3e-40ce-9d76-97cc86de0990 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:48.489320 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:48.568133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed7638-6a3e-40ce-9d76-97cc86de0990 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:48.600488 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7583a8d-ca20-43c4-a355-eca1c8399003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:48.601089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7583a8d-ca20-43c4-a355-eca1c8399003 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:48.601399 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7583a8d-ca20-43c4-a355-eca1c8399003 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:48.601638 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7583a8d-ca20-43c4-a355-eca1c8399003 None admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:48.602291 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7583a8d-ca20-43c4-a355-eca1c8399003 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:48.602592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7583a8d-ca20-43c4-a355-eca1c8399003 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:48.604035 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7583a8d-ca20-43c4-a355-eca1c8399003 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:48.612322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 634/1264] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:17:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:48.618072 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0585f4-416c-4626-a64f-1b14d835e96e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:48.683829 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:48.715165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0585f4-416c-4626-a64f-1b14d835e96e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:48.747777 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5ebcb-c735-42cb-b251-c0d6a9b71f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:48.748389 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5ebcb-c735-42cb-b251-c0d6a9b71f7b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:48.748665 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5ebcb-c735-42cb-b251-c0d6a9b71f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:48.748933 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5ebcb-c735-42cb-b251-c0d6a9b71f7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:48.758738 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a5ebcb-c735-42cb-b251-c0d6a9b71f7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:48.760162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a5ebcb-c735-42cb-b251-c0d6a9b71f7b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:48.773973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-a2a5ebcb-c735-42cb-b251-c0d6a9b71f7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:17:48.773973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:48.773973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:48.773973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:48.773973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:48.773973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:48.773973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:48.773973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:48.773973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:48.773973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:48.773973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:48.773973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:48.773973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:17:48.773973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:17:48.773973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:17:48.773973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:17:48.779792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:48.779792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:48.779792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:48.779792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:48.779792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:48.779792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:48.779792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:48.779792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:48.779792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:48.779792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:48.779792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:48.779792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:48.779792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:48.779792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:48.779792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:48.779792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:48.779792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:17:48.780727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:17:48.780727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:17:48.780727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:17:48.780727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 631/1265] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:17:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:17:48.781078 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7583a8d-ca20-43c4-a355-eca1c8399003 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:48.863613 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:48.882057 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7583a8d-ca20-43c4-a355-eca1c8399003 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:48.914610 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271248df-172c-49f0-80ea-4f1ab0d2cb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:48.915240 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271248df-172c-49f0-80ea-4f1ab0d2cb31 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:48.915520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271248df-172c-49f0-80ea-4f1ab0d2cb31 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:48.915791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271248df-172c-49f0-80ea-4f1ab0d2cb31 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:48.916428 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271248df-172c-49f0-80ea-4f1ab0d2cb31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:48.916730 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271248df-172c-49f0-80ea-4f1ab0d2cb31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:48.918075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271248df-172c-49f0-80ea-4f1ab0d2cb31 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:48.929857 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 635/1266] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:17:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:48.935551 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5ebcb-c735-42cb-b251-c0d6a9b71f7b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:49.034774 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5ebcb-c735-42cb-b251-c0d6a9b71f7b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:49.054190 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:49.068323 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0eda59-4022-453f-890c-336d5af4b53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:49.068944 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0eda59-4022-453f-890c-336d5af4b53e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:49.069217 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0eda59-4022-453f-890c-336d5af4b53e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:49.069484 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0eda59-4022-453f-890c-336d5af4b53e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:49.076506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0eda59-4022-453f-890c-336d5af4b53e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:49.077916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0eda59-4022-453f-890c-336d5af4b53e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:49.088561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-7c0eda59-4022-453f-890c-336d5af4b53e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 02 09:17:49.088561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:49.088561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:49.088561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:49.088561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:49.088561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:49.088561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:49.088561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:49.088561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:49.088561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:49.088561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:49.088561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:49.088561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:17:49.088561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:17:49.088561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:17:49.088561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:17:49.089689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:49.089689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:49.089689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:17:49.089689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:17:49.089689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:17:49.089689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:17:49.089689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:49.089689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:49.089689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:49.089689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:49.089689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:49.089689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:49.089689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:49.089689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:49.089689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:49.089689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:49.089689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:49.089689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:49.090721 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:49.090721 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:49.090721 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:17:49.090721 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:17:49.090721 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:17:49.090721 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:17:49.090721 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:17:49.090721 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:17:49.090721 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 02 09:17:49.090721 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:17:49.090721 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 632/1267] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:17:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:17:49.097718 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271248df-172c-49f0-80ea-4f1ab0d2cb31 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:49.193296 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271248df-172c-49f0-80ea-4f1ab0d2cb31 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:49.226035 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535a088-2519-4de4-b219-4cf3294354ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:49.226633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535a088-2519-4de4-b219-4cf3294354ec None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:49.226941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535a088-2519-4de4-b219-4cf3294354ec None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:49.227215 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535a088-2519-4de4-b219-4cf3294354ec None admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:49.227933 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535a088-2519-4de4-b219-4cf3294354ec None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:49.228238 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535a088-2519-4de4-b219-4cf3294354ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:49.229649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535a088-2519-4de4-b219-4cf3294354ec None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:49.238568 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:49.247677 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 636/1268] 10.222.0.57 () {56 vars in 1143 bytes} [Fri Sep 2 09:17:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:49.254003 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0eda59-4022-453f-890c-336d5af4b53e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:49.354875 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0eda59-4022-453f-890c-336d5af4b53e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:49.389044 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e25e5a7-c2ab-4d24-838c-caa45c34b95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:49.389673 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25e5a7-c2ab-4d24-838c-caa45c34b95f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:49.389985 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25e5a7-c2ab-4d24-838c-caa45c34b95f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:49.390262 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e25e5a7-c2ab-4d24-838c-caa45c34b95f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:49.400649 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e25e5a7-c2ab-4d24-838c-caa45c34b95f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:49.402099 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e25e5a7-c2ab-4d24-838c-caa45c34b95f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:49.416386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-7e25e5a7-c2ab-4d24-838c-caa45c34b95f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:17:49.416386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:49.416386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:49.416386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:49.416386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:49.416386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:49.416386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:49.416386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:49.416386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:49.416386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:49.416386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:49.416386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:49.416386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:17:49.416386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:17:49.416386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:17:49.416386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:17:49.417490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:49.417490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:49.417490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:49.417490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:49.417490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:49.417490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:49.417490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:49.417490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:49.417490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:49.417490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:49.417490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:49.417490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:49.417490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:49.417490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:49.417490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:49.417490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:49.417490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:17:49.418624 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:17:49.418624 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:17:49.418624 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:17:49.418624 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 633/1269] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:17:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:17:49.424137 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535a088-2519-4de4-b219-4cf3294354ec None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:49.527145 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535a088-2519-4de4-b219-4cf3294354ec None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:49.561156 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767db5e-0797-47da-9a60-769e60ea6a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:49.561758 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767db5e-0797-47da-9a60-769e60ea6a7a None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:49.562031 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767db5e-0797-47da-9a60-769e60ea6a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:49.562301 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767db5e-0797-47da-9a60-769e60ea6a7a None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:49.562960 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c767db5e-0797-47da-9a60-769e60ea6a7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:49.563260 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c767db5e-0797-47da-9a60-769e60ea6a7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:49.564630 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c767db5e-0797-47da-9a60-769e60ea6a7a None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:49.575470 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 637/1270] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:17:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:49.581303 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e25e5a7-c2ab-4d24-838c-caa45c34b95f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:49.613754 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:49.678779 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e25e5a7-c2ab-4d24-838c-caa45c34b95f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:49.710621 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ebd3b-7552-47ae-9ccc-8ad253a6af6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:49.711269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ebd3b-7552-47ae-9ccc-8ad253a6af6b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:49.711601 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ebd3b-7552-47ae-9ccc-8ad253a6af6b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:49.711890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ebd3b-7552-47ae-9ccc-8ad253a6af6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:49.718396 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1ebd3b-7552-47ae-9ccc-8ad253a6af6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:49.720342 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1ebd3b-7552-47ae-9ccc-8ad253a6af6b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:49.731754 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-bf1ebd3b-7552-47ae-9ccc-8ad253a6af6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:17:49.731754 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:49.731754 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:49.731754 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:49.731754 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:49.731754 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:49.731754 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:49.731754 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:49.731754 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:49.731754 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:49.731754 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:49.731754 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:49.731754 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:17:49.731754 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:17:49.731754 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:17:49.732815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:17:49.732815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:49.732815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:49.732815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:49.732815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:49.732815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:49.732815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:49.732815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:49.732815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:49.732815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:49.732815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:49.732815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:49.732815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:49.732815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:49.732815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:49.732815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:49.732815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:49.734008 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:17:49.734008 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:17:49.734008 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:17:49.734008 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:17:49.734008 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:17:49.734008 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:17:49.734008 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:17:49.734008 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:17:49.734008 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 634/1271] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:17:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:17:49.739296 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767db5e-0797-47da-9a60-769e60ea6a7a None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:49.796060 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:49.836012 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767db5e-0797-47da-9a60-769e60ea6a7a None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:49.869143 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93dc1cb-3598-4ab8-bfc8-4255dbfbca48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:49.869704 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93dc1cb-3598-4ab8-bfc8-4255dbfbca48 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:49.869996 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93dc1cb-3598-4ab8-bfc8-4255dbfbca48 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:49.870244 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93dc1cb-3598-4ab8-bfc8-4255dbfbca48 None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:49.873135 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93dc1cb-3598-4ab8-bfc8-4255dbfbca48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:49.873135 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93dc1cb-3598-4ab8-bfc8-4255dbfbca48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:49.873135 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93dc1cb-3598-4ab8-bfc8-4255dbfbca48 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:49.885067 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 638/1272] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:17:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:49.891205 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ebd3b-7552-47ae-9ccc-8ad253a6af6b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:49.981633 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:49.989861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ebd3b-7552-47ae-9ccc-8ad253a6af6b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:50.022226 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000261f5-f880-4826-99dc-534067282d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:50.022911 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000261f5-f880-4826-99dc-534067282d31 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:50.023200 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000261f5-f880-4826-99dc-534067282d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:50.023475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000261f5-f880-4826-99dc-534067282d31 None admin] PATH_INFO: `/v3/projects/d18a5117a03b4f21ab3290a15680de66/users/5177fa4d168c4c219e09c7178bbbc790/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:50.025360 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000261f5-f880-4826-99dc-534067282d31 None admin] RBAC: Authorizing `identity:create_grant(project_id=d18a5117a03b4f21ab3290a15680de66, user_id=5177fa4d168c4c219e09c7178bbbc790, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:50.026804 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000261f5-f880-4826-99dc-534067282d31 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:50.105255 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 635/1273] 10.222.0.57 () {58 vars in 1439 bytes} [Fri Sep 2 09:17:49 2022] PUT /identity/v3/projects/d18a5117a03b4f21ab3290a15680de66/users/5177fa4d168c4c219e09c7178bbbc790/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:17:52.003279 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d13ada-afb5-4b53-b182-1e673b4d701d None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:52.003841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d13ada-afb5-4b53-b182-1e673b4d701d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:52.003841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d13ada-afb5-4b53-b182-1e673b4d701d None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:52.005161 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 639/1274] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:17:52.010221 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18665390-256b-4143-af8f-a65dc683f2f4 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:52.010641 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18665390-256b-4143-af8f-a65dc683f2f4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:52.010812 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18665390-256b-4143-af8f-a65dc683f2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:52.029556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-18665390-256b-4143-af8f-a65dc683f2f4 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:52.030930 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-18665390-256b-4143-af8f-a65dc683f2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:52.096477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 636/1275] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:17:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 02 09:17:52.362447 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d13ada-afb5-4b53-b182-1e673b4d701d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:52.462202 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d13ada-afb5-4b53-b182-1e673b4d701d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:52.479315 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:52.496031 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d3713-dbdd-4885-995d-eb960e54049f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:52.496659 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d3713-dbdd-4885-995d-eb960e54049f None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:52.497015 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d3713-dbdd-4885-995d-eb960e54049f None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:52.497267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d3713-dbdd-4885-995d-eb960e54049f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:52.507599 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1d3713-dbdd-4885-995d-eb960e54049f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 02 09:17:52.512311 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ab1d3713-dbdd-4885-995d-eb960e54049f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:17:52.512311 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:52.512311 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:52.512311 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:52.512311 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:52.512311 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:52.512311 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:52.512311 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:52.512311 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:52.512311 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:52.512311 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:52.512311 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:52.512311 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 02 09:17:52.512311 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 02 09:17:52.512311 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 02 09:17:52.512311 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 02 09:17:52.513448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 02 09:17:52.513448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 02 09:17:52.513448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 02 09:17:52.513448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 02 09:17:52.513448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:52.513448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:52.513448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:52.513448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:52.513448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:52.513448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:52.513448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:52.513448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:52.513448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:52.513448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:52.513448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:52.513448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:52.514426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:52.514426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:52.514426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:52.514426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:52.514426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 02 09:17:52.514426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 02 09:17:52.514426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 02 09:17:52.514426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 02 09:17:52.514426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 02 09:17:52.514426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 02 09:17:52.514426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 02 09:17:52.514426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:17:52.514426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 640/1276] 10.222.0.57 () {56 vars in 1096 bytes} [Fri Sep 2 09:17:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 02 09:17:52.520292 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18665390-256b-4143-af8f-a65dc683f2f4 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:52.598673 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:52.620564 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18665390-256b-4143-af8f-a65dc683f2f4 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:52.649941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d584b226-0bc0-4942-92da-b46bdb21fa57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:52.650877 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d584b226-0bc0-4942-92da-b46bdb21fa57 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:52.650877 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d584b226-0bc0-4942-92da-b46bdb21fa57 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:52.651093 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d584b226-0bc0-4942-92da-b46bdb21fa57 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:52.651754 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d584b226-0bc0-4942-92da-b46bdb21fa57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:52.652054 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d584b226-0bc0-4942-92da-b46bdb21fa57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:52.653400 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d584b226-0bc0-4942-92da-b46bdb21fa57 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:52.661127 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 637/1277] 10.222.0.57 () {56 vars in 1099 bytes} [Fri Sep 2 09:17:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:52.666999 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d3713-dbdd-4885-995d-eb960e54049f None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:52.767822 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d3713-dbdd-4885-995d-eb960e54049f None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:52.801680 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e81b2-32dc-45b9-8a3d-b8b6ecb9620b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:52.802301 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e81b2-32dc-45b9-8a3d-b8b6ecb9620b None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:52.802580 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e81b2-32dc-45b9-8a3d-b8b6ecb9620b None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:52.802880 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e81b2-32dc-45b9-8a3d-b8b6ecb9620b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:52.812996 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e81b2-32dc-45b9-8a3d-b8b6ecb9620b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:52.815049 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e81b2-32dc-45b9-8a3d-b8b6ecb9620b None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:52.829629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-440e81b2-32dc-45b9-8a3d-b8b6ecb9620b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:17:52.829629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:52.829629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:52.829629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:52.829629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:52.829629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:52.829629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:52.829629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:52.829629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:52.829629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:52.829629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:52.829629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:52.829629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:17:52.829629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:17:52.829629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:17:52.829629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:17:52.840387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:52.840387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:52.840387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:52.840387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:52.840387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:52.840387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:52.840387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:52.840387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:52.840387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:52.840387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:52.840387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:52.840387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:52.840387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:52.840387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:52.840387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:52.840387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:52.840387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:17:52.841296 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:17:52.841296 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:17:52.841296 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:17:52.841296 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 641/1278] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:17:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:17:52.841521 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d584b226-0bc0-4942-92da-b46bdb21fa57 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:52.934367 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d584b226-0bc0-4942-92da-b46bdb21fa57 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:52.966799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b488611-35c3-4a59-a75e-59bfb6e63dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:52.967405 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b488611-35c3-4a59-a75e-59bfb6e63dca None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:52.967686 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b488611-35c3-4a59-a75e-59bfb6e63dca None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:52.967959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b488611-35c3-4a59-a75e-59bfb6e63dca None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:52.968602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b488611-35c3-4a59-a75e-59bfb6e63dca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:52.968910 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b488611-35c3-4a59-a75e-59bfb6e63dca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:52.970397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b488611-35c3-4a59-a75e-59bfb6e63dca None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:52.981900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 638/1279] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:52.988268 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e81b2-32dc-45b9-8a3d-b8b6ecb9620b None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:53.021139 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:53.087523 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e81b2-32dc-45b9-8a3d-b8b6ecb9620b None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:53.119657 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac195d02-f979-4b1f-be67-9de917874fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:53.120276 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac195d02-f979-4b1f-be67-9de917874fbe None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:53.120557 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac195d02-f979-4b1f-be67-9de917874fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:53.120826 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac195d02-f979-4b1f-be67-9de917874fbe None admin] PATH_INFO: `/v3/users/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:53.128630 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac195d02-f979-4b1f-be67-9de917874fbe None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:53.130049 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac195d02-f979-4b1f-be67-9de917874fbe None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:53.140984 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ac195d02-f979-4b1f-be67-9de917874fbe None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 02 09:17:53.140984 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:53.140984 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:53.140984 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:53.140984 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:53.140984 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:53.140984 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:53.140984 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:53.140984 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:53.140984 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:53.140984 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:53.140984 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:53.140984 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 02 09:17:53.140984 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 02 09:17:53.140984 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 02 09:17:53.140984 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 02 09:17:53.145188 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:53.145188 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:53.145188 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 02 09:17:53.145188 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:17:53.145188 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 02 09:17:53.145188 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 02 09:17:53.145188 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:53.145188 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:53.145188 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:53.145188 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:53.145188 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:53.145188 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:53.145188 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:53.145188 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:53.145188 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:53.145188 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:53.145188 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:53.145188 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:53.146394 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:53.146394 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:53.146394 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 02 09:17:53.146394 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 02 09:17:53.146394 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 02 09:17:53.146394 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 02 09:17:53.146394 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 02 09:17:53.146394 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 02 09:17:53.146394 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 02 09:17:53.146394 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:17:53.146394 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 642/1280] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:17:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:17:53.147940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b488611-35c3-4a59-a75e-59bfb6e63dca None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:53.151700 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:53.243170 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b488611-35c3-4a59-a75e-59bfb6e63dca None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:53.278061 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33181acc-f3d1-424c-823f-4330b7d31c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:53.278660 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33181acc-f3d1-424c-823f-4330b7d31c6a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:53.278966 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33181acc-f3d1-424c-823f-4330b7d31c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:53.279233 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33181acc-f3d1-424c-823f-4330b7d31c6a None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:53.279944 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33181acc-f3d1-424c-823f-4330b7d31c6a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:53.280247 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33181acc-f3d1-424c-823f-4330b7d31c6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:53.281655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33181acc-f3d1-424c-823f-4330b7d31c6a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:53.298956 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 639/1281] 10.222.0.57 () {56 vars in 1143 bytes} [Fri Sep 2 09:17:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:17:53.304786 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac195d02-f979-4b1f-be67-9de917874fbe None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:53.394088 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:53.403878 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac195d02-f979-4b1f-be67-9de917874fbe None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:53.437398 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483e897-94c6-4c61-bf39-8e293acf61de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:53.438036 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483e897-94c6-4c61-bf39-8e293acf61de None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:53.438377 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483e897-94c6-4c61-bf39-8e293acf61de None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:53.438688 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0483e897-94c6-4c61-bf39-8e293acf61de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:53.504154 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0483e897-94c6-4c61-bf39-8e293acf61de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:53.505517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0483e897-94c6-4c61-bf39-8e293acf61de None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:53.591721 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:53.602988 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 643/1282] 10.222.0.57 () {58 vars in 1282 bytes} [Fri Sep 2 09:17:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 02 09:17:53.608650 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33181acc-f3d1-424c-823f-4330b7d31c6a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:53.669938 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:53.706974 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33181acc-f3d1-424c-823f-4330b7d31c6a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:53.738850 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1c9ad-252c-4658-8eb5-c80af1fc0b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:53.739465 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1c9ad-252c-4658-8eb5-c80af1fc0b92 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:53.739741 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1c9ad-252c-4658-8eb5-c80af1fc0b92 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:53.740011 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1c9ad-252c-4658-8eb5-c80af1fc0b92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:53.749431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1c9ad-252c-4658-8eb5-c80af1fc0b92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:53.750903 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1c9ad-252c-4658-8eb5-c80af1fc0b92 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:53.764917 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-b2a1c9ad-252c-4658-8eb5-c80af1fc0b92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:17:53.764917 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:53.764917 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:53.764917 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:53.764917 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:53.764917 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:53.764917 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:53.764917 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:53.764917 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:53.764917 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:53.764917 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:53.764917 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:53.764917 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 02 09:17:53.764917 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 02 09:17:53.764917 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 02 09:17:53.764917 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 02 09:17:53.766057 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:53.766057 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:53.766057 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:53.766057 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:53.766057 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:53.766057 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:53.766057 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:53.766057 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:53.766057 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:53.766057 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:53.766057 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:53.766057 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:53.766057 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:53.766057 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:53.766057 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:53.766057 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:53.766057 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 02 09:17:53.767497 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 02 09:17:53.767497 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 02 09:17:53.767497 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:17:53.767497 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 640/1283] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:17:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:17:53.772370 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483e897-94c6-4c61-bf39-8e293acf61de None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:53.869306 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0483e897-94c6-4c61-bf39-8e293acf61de None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:53.903214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48883e9-6bdc-432b-8c03-32778cf037c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:53.903811 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48883e9-6bdc-432b-8c03-32778cf037c1 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:53.904086 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48883e9-6bdc-432b-8c03-32778cf037c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:53.904353 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48883e9-6bdc-432b-8c03-32778cf037c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:53.904985 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48883e9-6bdc-432b-8c03-32778cf037c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:53.905285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48883e9-6bdc-432b-8c03-32778cf037c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:53.906650 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48883e9-6bdc-432b-8c03-32778cf037c1 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:53.918070 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 644/1284] 10.222.0.57 () {56 vars in 1109 bytes} [Fri Sep 2 09:17:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:53.923785 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1c9ad-252c-4658-8eb5-c80af1fc0b92 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:54.021799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1c9ad-252c-4658-8eb5-c80af1fc0b92 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:54.034964 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:54.053906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8407d-d910-45a4-a503-96dabab3c3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:54.054556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8407d-d910-45a4-a503-96dabab3c3a9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:54.054872 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8407d-d910-45a4-a503-96dabab3c3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:54.055146 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8407d-d910-45a4-a503-96dabab3c3a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:54.061750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec8407d-d910-45a4-a503-96dabab3c3a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:54.063713 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec8407d-d910-45a4-a503-96dabab3c3a9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:54.074658 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-1ec8407d-d910-45a4-a503-96dabab3c3a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:17:54.074658 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:54.074658 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:54.074658 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:54.074658 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:54.074658 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:54.074658 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:54.074658 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:54.074658 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:54.074658 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:54.074658 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:54.074658 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:54.074658 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 02 09:17:54.074658 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 02 09:17:54.074658 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 02 09:17:54.075813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 02 09:17:54.075813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:54.075813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:54.075813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:54.075813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:54.075813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:54.075813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:54.075813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:54.075813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:54.075813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:54.075813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:54.075813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:54.075813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:54.075813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:54.075813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:54.075813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:54.075813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:54.077186 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:17:54.077186 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:17:54.077186 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:17:54.077186 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:17:54.077186 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:17:54.077186 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:17:54.077186 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 02 09:17:54.077186 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:17:54.077186 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 641/1285] 10.222.0.57 () {56 vars in 1111 bytes} [Fri Sep 2 09:17:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:17:54.082257 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48883e9-6bdc-432b-8c03-32778cf037c1 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:54.131387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:54.179195 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48883e9-6bdc-432b-8c03-32778cf037c1 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:54.212160 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f27014-3ae7-4875-ae01-4f0fe953542d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:54.212762 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f27014-3ae7-4875-ae01-4f0fe953542d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:54.213042 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f27014-3ae7-4875-ae01-4f0fe953542d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:54.213313 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f27014-3ae7-4875-ae01-4f0fe953542d None admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:54.213992 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f27014-3ae7-4875-ae01-4f0fe953542d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:54.214298 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f27014-3ae7-4875-ae01-4f0fe953542d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:54.215725 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f27014-3ae7-4875-ae01-4f0fe953542d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:54.228086 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 645/1286] 10.222.0.57 () {56 vars in 1148 bytes} [Fri Sep 2 09:17:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:54.234220 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8407d-d910-45a4-a503-96dabab3c3a9 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:54.336629 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8407d-d910-45a4-a503-96dabab3c3a9 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:54.369571 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830809f-51b3-4588-a512-993b33614301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:54.370177 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830809f-51b3-4588-a512-993b33614301 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:54.370444 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830809f-51b3-4588-a512-993b33614301 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:54.370739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830809f-51b3-4588-a512-993b33614301 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:54.371453 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1830809f-51b3-4588-a512-993b33614301 None admin] RBAC: Adding query filter params (role.id=f886b8e8889549e896a8fb3426cce9e1, scope.project.id=d18a5117a03b4f21ab3290a15680de66, user.id=5177fa4d168c4c219e09c7178bbbc790) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:54.371761 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1830809f-51b3-4588-a512-993b33614301 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:54.373168 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1830809f-51b3-4588-a512-993b33614301 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:54.379405 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 642/1287] 10.222.0.57 () {56 vars in 1374 bytes} [Fri Sep 2 09:17:54 2022] GET /identity/v3/role_assignments?user.id=5177fa4d168c4c219e09c7178bbbc790&scope.project.id=d18a5117a03b4f21ab3290a15680de66&role.id=f886b8e8889549e896a8fb3426cce9e1 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:56.271761 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439faa9-c826-4187-abe7-348c1eb61afe None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:56.272323 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439faa9-c826-4187-abe7-348c1eb61afe None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:56.272323 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439faa9-c826-4187-abe7-348c1eb61afe None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:56.273552 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 646/1288] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:17:56.278579 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cd81a-84fe-4ed1-a203-1174e102106c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:56.278989 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cd81a-84fe-4ed1-a203-1174e102106c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:56.279167 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cd81a-84fe-4ed1-a203-1174e102106c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:56.297278 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a22cd81a-84fe-4ed1-a203-1174e102106c None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:56.300906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a22cd81a-84fe-4ed1-a203-1174e102106c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:56.329845 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:56.365484 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 643/1289] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:17:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:17:56.630126 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439faa9-c826-4187-abe7-348c1eb61afe None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:56.730391 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439faa9-c826-4187-abe7-348c1eb61afe None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:56.741006 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:56.763339 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc71675-3432-4c31-ac2d-064170c51d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:56.763956 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc71675-3432-4c31-ac2d-064170c51d5d None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:56.764231 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc71675-3432-4c31-ac2d-064170c51d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:56.764500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc71675-3432-4c31-ac2d-064170c51d5d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:56.765067 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc71675-3432-4c31-ac2d-064170c51d5d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:56.766998 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc71675-3432-4c31-ac2d-064170c51d5d None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:56.775639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-0bc71675-3432-4c31-ac2d-064170c51d5d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 02 09:17:56.775639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:56.775639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 09:17:56.775639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 09:17:56.775639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 09:17:56.775639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 09:17:56.775639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 09:17:56.775639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:17:56.775639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 02 09:17:56.775639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:17:56.775639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:17:56.775639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:17:56.775639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:17:56.775639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:17:56.775639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:17:56.775639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:17:56.775639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:17:56.783759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:17:56.783759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:17:56.783759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:17:56.783759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:17:56.783759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:17:56.783759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:17:56.783759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 09:17:56.783759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 09:17:56.783759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 09:17:56.783759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 09:17:56.783759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:17:56.783759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:17:56.783759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:17:56.783759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:17:56.783759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:17:56.783759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:17:56.784843 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:17:56.784843 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:17:56.784843 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:17:56.784843 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:17:56.784843 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:17:56.784843 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:17:56.784843 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:17:56.784843 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:17:56.784843 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:17:56.784843 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:17:56.784843 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 09:17:56.784843 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:17:56.784843 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 02 09:17:56.784843 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:17:56.784843 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 647/1290] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Sep 2 09:17:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:17:56.785884 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22cd81a-84fe-4ed1-a203-1174e102106c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:56.881979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22cd81a-84fe-4ed1-a203-1174e102106c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:56.914935 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e7f63-e141-42c1-9454-6b03941eaa5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:56.915537 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e7f63-e141-42c1-9454-6b03941eaa5e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:56.915816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e7f63-e141-42c1-9454-6b03941eaa5e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:56.916088 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e7f63-e141-42c1-9454-6b03941eaa5e None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:56.916713 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e7f63-e141-42c1-9454-6b03941eaa5e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:56.917014 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e7f63-e141-42c1-9454-6b03941eaa5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:56.918363 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e7f63-e141-42c1-9454-6b03941eaa5e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:56.923098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 644/1291] 10.222.0.57 () {56 vars in 1116 bytes} [Fri Sep 2 09:17:56 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:56.928802 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc71675-3432-4c31-ac2d-064170c51d5d None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:57.034344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc71675-3432-4c31-ac2d-064170c51d5d None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:57.071258 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab9f43-a29f-4f87-b556-5b7489353cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:57.071883 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab9f43-a29f-4f87-b556-5b7489353cac None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:57.072160 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab9f43-a29f-4f87-b556-5b7489353cac None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:57.072432 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab9f43-a29f-4f87-b556-5b7489353cac None admin] PATH_INFO: `/v3/services` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:57.073069 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab9f43-a29f-4f87-b556-5b7489353cac None admin] RBAC: Adding query filter params (type=placement) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:17:57.073373 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab9f43-a29f-4f87-b556-5b7489353cac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:57.074754 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab9f43-a29f-4f87-b556-5b7489353cac None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:57.080214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 648/1292] 10.222.0.57 () {56 vars in 1116 bytes} [Fri Sep 2 09:17:56 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:17:59.034513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761afde-55c7-45e4-b266-63a0db84cae6 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:59.035102 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761afde-55c7-45e4-b266-63a0db84cae6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:59.035102 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761afde-55c7-45e4-b266-63a0db84cae6 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:59.036438 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 645/1293] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:17:59.041776 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecc937-2b07-4044-87cf-79e50b76af78 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:59.042213 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecc937-2b07-4044-87cf-79e50b76af78 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:59.042308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ecc937-2b07-4044-87cf-79e50b76af78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:59.060548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-32ecc937-2b07-4044-87cf-79e50b76af78 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:17:59.061726 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-32ecc937-2b07-4044-87cf-79e50b76af78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:17:59.090671 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:59.129089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 649/1294] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:17:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:17:59.399297 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761afde-55c7-45e4-b266-63a0db84cae6 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:17:59.429531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:17:59.498716 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761afde-55c7-45e4-b266-63a0db84cae6 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:17:59.532958 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5a0d4d-d612-4785-b6a0-6513ebd4ca6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:17:59.533556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a0d4d-d612-4785-b6a0-6513ebd4ca6c None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:17:59.533835 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a0d4d-d612-4785-b6a0-6513ebd4ca6c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:17:59.534106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5a0d4d-d612-4785-b6a0-6513ebd4ca6c None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:17:59.534765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a0d4d-d612-4785-b6a0-6513ebd4ca6c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:17:59.536115 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5a0d4d-d612-4785-b6a0-6513ebd4ca6c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:17:59.666596 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 646/1295] 10.222.0.57 () {60 vars in 1141 bytes} [Fri Sep 2 09:17:59 2022] POST /identity/v3/services => generated 246 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:18:01.571831 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0b7ec-e9f6-4801-9b98-84744e31a4d0 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:18:01.572355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0b7ec-e9f6-4801-9b98-84744e31a4d0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:18:01.572355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0b7ec-e9f6-4801-9b98-84744e31a4d0 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:18:01.573788 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 650/1296] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:18:01.579632 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9b7cf-c8ee-4219-af23-6c38e520afac None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:18:01.580048 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9b7cf-c8ee-4219-af23-6c38e520afac None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:18:01.580194 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d9b7cf-c8ee-4219-af23-6c38e520afac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:18:01.597804 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c0d9b7cf-c8ee-4219-af23-6c38e520afac None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:18:01.599014 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c0d9b7cf-c8ee-4219-af23-6c38e520afac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:18:01.669384 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 647/1297] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:18:01 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:18:01.936978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0b7ec-e9f6-4801-9b98-84744e31a4d0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:18:02.036758 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0b7ec-e9f6-4801-9b98-84744e31a4d0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:18:02.070569 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50efc6-cc99-4511-9cfe-65d891c2c437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:18:02.071208 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50efc6-cc99-4511-9cfe-65d891c2c437 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:18:02.071483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50efc6-cc99-4511-9cfe-65d891c2c437 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:18:02.071750 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50efc6-cc99-4511-9cfe-65d891c2c437 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:18:02.072366 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c50efc6-cc99-4511-9cfe-65d891c2c437 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:18:02.073711 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c50efc6-cc99-4511-9cfe-65d891c2c437 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:18:02.082632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-5c50efc6-cc99-4511-9cfe-65d891c2c437 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 02 09:18:02.082632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:18:02.082632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 09:18:02.082632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 09:18:02.082632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 09:18:02.082632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 09:18:02.082632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 09:18:02.082632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:18:02.082632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 02 09:18:02.082632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:18:02.082632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:18:02.082632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:18:02.082632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:18:02.082632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:18:02.082632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:18:02.082632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:18:02.082632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:18:02.083946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:18:02.083946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:18:02.083946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:18:02.083946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:18:02.083946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:18:02.083946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:18:02.083946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 09:18:02.083946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 09:18:02.083946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 09:18:02.083946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 09:18:02.083946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:18:02.083946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:18:02.083946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:18:02.083946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:18:02.083946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:18:02.083946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:18:02.085077 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:18:02.085077 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:18:02.085077 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:18:02.085077 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:18:02.085077 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:18:02.085077 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:18:02.085077 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:18:02.085077 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:18:02.085077 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:18:02.085077 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:18:02.085077 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 09:18:02.085077 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:18:02.085077 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 02 09:18:02.085077 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:18:02.085077 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 651/1298] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Sep 2 09:18:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:18:02.089767 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9b7cf-c8ee-4219-af23-6c38e520afac None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:18:02.107694 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:18:02.185895 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d9b7cf-c8ee-4219-af23-6c38e520afac None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:18:02.219426 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528a576-67a1-487e-a003-757ef50e12fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:18:02.220045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528a576-67a1-487e-a003-757ef50e12fd None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:18:02.220322 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528a576-67a1-487e-a003-757ef50e12fd None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:18:02.220595 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528a576-67a1-487e-a003-757ef50e12fd None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:18:02.221213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528a576-67a1-487e-a003-757ef50e12fd None admin] RBAC: Adding query filter params (name=placement) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:18:02.221510 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528a576-67a1-487e-a003-757ef50e12fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:18:02.222886 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528a576-67a1-487e-a003-757ef50e12fd None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:18:02.227815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 648/1299] 10.222.0.57 () {56 vars in 1116 bytes} [Fri Sep 2 09:18:02 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:18:02.233834 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50efc6-cc99-4511-9cfe-65d891c2c437 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:18:02.265183 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:18:02.331822 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50efc6-cc99-4511-9cfe-65d891c2c437 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:18:02.364929 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf82098-a07d-4233-807b-1967368eefa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:18:02.365495 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf82098-a07d-4233-807b-1967368eefa9 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:18:02.365774 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf82098-a07d-4233-807b-1967368eefa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:18:02.366089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf82098-a07d-4233-807b-1967368eefa9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:18:02.367008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf82098-a07d-4233-807b-1967368eefa9 None admin] RBAC: Adding query filter params (interface=public, service_id=a07dcccfe4614246bb5a9cbaa7f64715, region_id=RegionOne) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:18:02.367314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf82098-a07d-4233-807b-1967368eefa9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:18:02.368663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf82098-a07d-4233-807b-1967368eefa9 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:18:02.373432 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 652/1300] 10.222.0.57 () {56 vars in 1251 bytes} [Fri Sep 2 09:18:02 2022] GET /identity/v3/endpoints?service_id=a07dcccfe4614246bb5a9cbaa7f64715&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:18:02.379123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528a576-67a1-487e-a003-757ef50e12fd None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:18:02.458462 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:18:02.476519 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528a576-67a1-487e-a003-757ef50e12fd None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:18:02.509650 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195a13e-8efe-4481-b5f2-01dfa6bea6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:18:02.510257 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195a13e-8efe-4481-b5f2-01dfa6bea6c8 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:18:02.510588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195a13e-8efe-4481-b5f2-01dfa6bea6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:18:02.510887 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195a13e-8efe-4481-b5f2-01dfa6bea6c8 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:18:02.511490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e195a13e-8efe-4481-b5f2-01dfa6bea6c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:18:02.512837 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e195a13e-8efe-4481-b5f2-01dfa6bea6c8 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:18:02.519892 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 649/1301] 10.222.0.57 () {56 vars in 1087 bytes} [Fri Sep 2 09:18:02 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:18:04.410022 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b191c-6831-4c85-9b32-61ad9967550b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:18:04.410570 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b191c-6831-4c85-9b32-61ad9967550b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:18:04.410570 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b191c-6831-4c85-9b32-61ad9967550b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:18:04.411964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 653/1302] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:18:04.416954 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6ac5d-b244-43a3-ac74-3db6f9238b42 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:18:04.417332 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6ac5d-b244-43a3-ac74-3db6f9238b42 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:18:04.417508 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6ac5d-b244-43a3-ac74-3db6f9238b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:18:04.437281 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6bb6ac5d-b244-43a3-ac74-3db6f9238b42 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:18:04.439046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6bb6ac5d-b244-43a3-ac74-3db6f9238b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:18:04.503500 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 650/1303] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:18:04 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 02 09:18:04.770369 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b191c-6831-4c85-9b32-61ad9967550b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:18:04.865554 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:18:04.872150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b191c-6831-4c85-9b32-61ad9967550b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:18:04.905317 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbdc017-5878-4b0d-b916-7cf20bdc6e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:18:04.905943 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdc017-5878-4b0d-b916-7cf20bdc6e03 None admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:18:04.906263 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdc017-5878-4b0d-b916-7cf20bdc6e03 None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:18:04.906541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdc017-5878-4b0d-b916-7cf20bdc6e03 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:18:04.907158 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbdc017-5878-4b0d-b916-7cf20bdc6e03 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:18:04.908500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbdc017-5878-4b0d-b916-7cf20bdc6e03 None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:18:04.917047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application [None req-ebbdc017-5878-4b0d-b916-7cf20bdc6e03 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 02 09:18:04.917047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:18:04.917047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 02 09:18:04.917047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 02 09:18:04.917047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 02 09:18:04.917047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 02 09:18:04.917047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 02 09:18:04.917047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:18:04.917047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 02 09:18:04.917047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:18:04.917047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 02 09:18:04.917047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:18:04.917047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:18:04.917047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:18:04.917047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:18:04.917047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:18:04.917047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:18:04.918241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:18:04.918241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:18:04.918241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:18:04.918241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:18:04.918241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:18:04.918241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:18:04.918241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 02 09:18:04.918241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 02 09:18:04.918241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 02 09:18:04.918241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 02 09:18:04.918241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:18:04.918241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:18:04.918241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:18:04.918241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:18:04.918241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:18:04.918241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:18:04.919357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:18:04.919357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application with Lock( Sep 02 09:18:04.919357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:18:04.919357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:18:04.919357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:18:04.919357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:18:04.919357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:18:04.919357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:18:04.919357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:18:04.919357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:18:04.919357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 02 09:18:04.919357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 02 09:18:04.919357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 02 09:18:04.919357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: ERROR keystone.server.flask.application Sep 02 09:18:04.919357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 654/1304] 10.222.0.57 () {56 vars in 1117 bytes} [Fri Sep 2 09:18:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 02 09:18:04.924639 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb6ac5d-b244-43a3-ac74-3db6f9238b42 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:18:04.998682 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:18:05.021145 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb6ac5d-b244-43a3-ac74-3db6f9238b42 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:18:05.053942 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e0edb6-5355-4d00-a797-6980b36a5b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:18:05.054624 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0edb6-5355-4d00-a797-6980b36a5b1a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:18:05.054953 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0edb6-5355-4d00-a797-6980b36a5b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:18:05.055375 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0edb6-5355-4d00-a797-6980b36a5b1a None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:18:05.056010 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e0edb6-5355-4d00-a797-6980b36a5b1a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:18:05.056320 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e0edb6-5355-4d00-a797-6980b36a5b1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:18:05.057675 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e0edb6-5355-4d00-a797-6980b36a5b1a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:18:05.063641 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 651/1305] 10.222.0.57 () {56 vars in 1116 bytes} [Fri Sep 2 09:18:04 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:18:05.069658 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbdc017-5878-4b0d-b916-7cf20bdc6e03 None admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:18:05.168516 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbdc017-5878-4b0d-b916-7cf20bdc6e03 None admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:18:05.201352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0df2ef-7a1c-4685-be7b-406912a48dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:18:05.201946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0df2ef-7a1c-4685-be7b-406912a48dec None admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:18:05.202224 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0df2ef-7a1c-4685-be7b-406912a48dec None admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:18:05.202493 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0df2ef-7a1c-4685-be7b-406912a48dec None admin] PATH_INFO: `/v3/endpoints` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:18:05.203216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0df2ef-7a1c-4685-be7b-406912a48dec None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:18:05.204562 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0df2ef-7a1c-4685-be7b-406912a48dec None admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:18:05.317461 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 655/1306] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:18:05 2022] POST /identity/v3/endpoints => generated 331 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:18:15.070783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=30 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:18:15.312995 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:24:22.716089 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d426f-bcba-4b29-8a0c-a01e87d6d6ff None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:22.716569 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d426f-bcba-4b29-8a0c-a01e87d6d6ff None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:22.716569 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d426f-bcba-4b29-8a0c-a01e87d6d6ff None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:22.717905 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 652/1307] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:24:22.725872 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de4a8c-a6b9-4f66-bfbd-1223c2310cce None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:22.727047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de4a8c-a6b9-4f66-bfbd-1223c2310cce None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:22.728371 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de4a8c-a6b9-4f66-bfbd-1223c2310cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:22.880667 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-91de4a8c-a6b9-4f66-bfbd-1223c2310cce None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:24:22.880667 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-91de4a8c-a6b9-4f66-bfbd-1223c2310cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:23.063256 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 656/1308] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:24:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:24:23.367435 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b482770-4171-4f49-93eb-7b9afc4d2309 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:23.368640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b482770-4171-4f49-93eb-7b9afc4d2309 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:23.369122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b482770-4171-4f49-93eb-7b9afc4d2309 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:23.371167 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 653/1309] 10.222.0.57 () {54 vars in 965 bytes} [Fri Sep 2 09:24:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:24:23.377765 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f0b5f-553e-4950-99c8-bf10884c7181 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:23.378058 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f0b5f-553e-4950-99c8-bf10884c7181 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:23.378340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f0b5f-553e-4950-99c8-bf10884c7181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:23.463722 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-fc8f0b5f-553e-4950-99c8-bf10884c7181 None None] MFA Rules not processed for user `dfea2ee0c9e84b48ac9bafe3173d937a`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:24:23.465357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fc8f0b5f-553e-4950-99c8-bf10884c7181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:23.488545 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:24:23.571053 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 657/1310] 10.222.0.57 () {58 vars in 1062 bytes} [Fri Sep 2 09:24:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:24:23.580631 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b482770-4171-4f49-93eb-7b9afc4d2309 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:24:23.582889 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9b482770-4171-4f49-93eb-7b9afc4d2309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:23.712076 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b482770-4171-4f49-93eb-7b9afc4d2309 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:24:23.793149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677c945-2d7a-4b7b-96fe-4aae80677517 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:24:23.793784 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677c945-2d7a-4b7b-96fe-4aae80677517 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:23.794029 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677c945-2d7a-4b7b-96fe-4aae80677517 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:23.794429 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677c945-2d7a-4b7b-96fe-4aae80677517 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:23.796063 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e677c945-2d7a-4b7b-96fe-4aae80677517 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:23.857144 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:24:23.916093 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677c945-2d7a-4b7b-96fe-4aae80677517 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:24:23.917561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677c945-2d7a-4b7b-96fe-4aae80677517 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:24:23.937823 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 654/1311] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:24:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:24:26.203560 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b012a-d7a2-45e9-81a4-fa5ff61d2261 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:26.204059 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b012a-d7a2-45e9-81a4-fa5ff61d2261 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:26.204156 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b012a-d7a2-45e9-81a4-fa5ff61d2261 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:26.205474 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 658/1312] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:24:26.210634 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63849aec-56ba-4482-b2a9-c29aee531a95 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:26.211084 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63849aec-56ba-4482-b2a9-c29aee531a95 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:26.211212 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63849aec-56ba-4482-b2a9-c29aee531a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:26.280559 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-63849aec-56ba-4482-b2a9-c29aee531a95 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:24:26.281764 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-63849aec-56ba-4482-b2a9-c29aee531a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:26.380683 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 655/1313] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:24:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:24:26.663557 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b012a-d7a2-45e9-81a4-fa5ff61d2261 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:24:26.665684 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-724b012a-d7a2-45e9-81a4-fa5ff61d2261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:26.777463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b012a-d7a2-45e9-81a4-fa5ff61d2261 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:24:26.847278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:24:26.849798 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d58597-a9df-46dc-821f-1df2b1f47788 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:24:26.850405 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d58597-a9df-46dc-821f-1df2b1f47788 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:26.850682 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d58597-a9df-46dc-821f-1df2b1f47788 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:26.850977 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d58597-a9df-46dc-821f-1df2b1f47788 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:26.853204 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-76d58597-a9df-46dc-821f-1df2b1f47788 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:26.961522 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d58597-a9df-46dc-821f-1df2b1f47788 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:24:26.962976 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d58597-a9df-46dc-821f-1df2b1f47788 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:24:26.981415 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 659/1314] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:24:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:24:29.381220 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10e195-1da6-4a5a-a3f1-10517b4dc4a6 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:29.381737 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10e195-1da6-4a5a-a3f1-10517b4dc4a6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:29.381929 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c10e195-1da6-4a5a-a3f1-10517b4dc4a6 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:29.383678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 656/1315] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:24:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:24:29.388775 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d58395-e0d2-4485-9bfd-deadb378ae13 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:29.389146 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d58395-e0d2-4485-9bfd-deadb378ae13 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:29.389312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d58395-e0d2-4485-9bfd-deadb378ae13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:29.454844 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f6d58395-e0d2-4485-9bfd-deadb378ae13 None None] MFA Rules not processed for user `604723d5b48f4e869bccbe56651aa80b`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:24:29.456143 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f6d58395-e0d2-4485-9bfd-deadb378ae13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:29.507576 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:24:29.551472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 660/1316] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:24:29 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:24:29.836622 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10e195-1da6-4a5a-a3f1-10517b4dc4a6 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:24:29.838153 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5c10e195-1da6-4a5a-a3f1-10517b4dc4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:29.848056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:24:29.942431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c10e195-1da6-4a5a-a3f1-10517b4dc4a6 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:24:30.013213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b96f89-c96e-4cb1-a5ad-c380fca0efd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:24:30.013818 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b96f89-c96e-4cb1-a5ad-c380fca0efd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:30.014098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b96f89-c96e-4cb1-a5ad-c380fca0efd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:30.014370 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b96f89-c96e-4cb1-a5ad-c380fca0efd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:30.015997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-79b96f89-c96e-4cb1-a5ad-c380fca0efd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:30.076325 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:24:30.122567 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b96f89-c96e-4cb1-a5ad-c380fca0efd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:24:30.124019 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b96f89-c96e-4cb1-a5ad-c380fca0efd6 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:24:30.141174 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 657/1317] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:24:29 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:24:32.832765 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353dc1d8-e97f-4828-8360-5bce18a45fdd None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:32.833289 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353dc1d8-e97f-4828-8360-5bce18a45fdd None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:32.833289 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353dc1d8-e97f-4828-8360-5bce18a45fdd None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:32.834601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 661/1318] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:24:32.840170 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afebfd-ea99-44cb-b181-44a9f171e36b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:32.840597 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afebfd-ea99-44cb-b181-44a9f171e36b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:32.840732 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afebfd-ea99-44cb-b181-44a9f171e36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:32.905752 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a4afebfd-ea99-44cb-b181-44a9f171e36b None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:24:32.907044 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a4afebfd-ea99-44cb-b181-44a9f171e36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:32.999550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 658/1319] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:24:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:24:33.279804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353dc1d8-e97f-4828-8360-5bce18a45fdd None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:24:33.281337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-353dc1d8-e97f-4828-8360-5bce18a45fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:33.382317 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353dc1d8-e97f-4828-8360-5bce18a45fdd None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:24:33.454022 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ee245-1078-4050-b94a-7328b966fa12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:24:33.454624 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ee245-1078-4050-b94a-7328b966fa12 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:33.454935 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ee245-1078-4050-b94a-7328b966fa12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:33.455211 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ee245-1078-4050-b94a-7328b966fa12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:33.456785 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-226ee245-1078-4050-b94a-7328b966fa12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:33.464552 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:24:33.562956 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ee245-1078-4050-b94a-7328b966fa12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:24:33.564315 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ee245-1078-4050-b94a-7328b966fa12 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:24:33.584819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 662/1320] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:24:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:24:36.697838 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56887889-786a-4a8d-87ba-db1581bc0e17 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:36.698386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56887889-786a-4a8d-87ba-db1581bc0e17 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:36.698386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56887889-786a-4a8d-87ba-db1581bc0e17 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:36.699804 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 659/1321] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:24:36.705117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010272f-52d9-432e-ba62-0f62092af75f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:36.705539 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010272f-52d9-432e-ba62-0f62092af75f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:36.705661 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010272f-52d9-432e-ba62-0f62092af75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:36.770915 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d010272f-52d9-432e-ba62-0f62092af75f None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:24:36.772118 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d010272f-52d9-432e-ba62-0f62092af75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:36.828658 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:24:36.869977 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 663/1322] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:24:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:24:37.169851 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08141bc6-ae14-4614-b3e6-757cba3cbd05 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:37.170218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08141bc6-ae14-4614-b3e6-757cba3cbd05 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:37.170554 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08141bc6-ae14-4614-b3e6-757cba3cbd05 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:37.172034 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 660/1323] 10.222.0.57 () {54 vars in 965 bytes} [Fri Sep 2 09:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:24:37.178082 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2451827-a539-493b-86c9-aaf9f0fb4289 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:37.178370 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2451827-a539-493b-86c9-aaf9f0fb4289 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:37.178639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2451827-a539-493b-86c9-aaf9f0fb4289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:37.253467 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b2451827-a539-493b-86c9-aaf9f0fb4289 None None] MFA Rules not processed for user `dfea2ee0c9e84b48ac9bafe3173d937a`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:24:37.254675 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b2451827-a539-493b-86c9-aaf9f0fb4289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:37.328362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 664/1324] 10.222.0.57 () {58 vars in 1062 bytes} [Fri Sep 2 09:24:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:24:37.338080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08141bc6-ae14-4614-b3e6-757cba3cbd05 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:24:37.339669 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-08141bc6-ae14-4614-b3e6-757cba3cbd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:37.360217 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:24:37.442645 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08141bc6-ae14-4614-b3e6-757cba3cbd05 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:24:37.515960 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff7aa16-b0c7-45c4-886b-d258c1c2278f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:24:37.516749 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7aa16-b0c7-45c4-886b-d258c1c2278f service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:37.516890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7aa16-b0c7-45c4-886b-d258c1c2278f service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:37.517112 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff7aa16-b0c7-45c4-886b-d258c1c2278f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:37.518686 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fff7aa16-b0c7-45c4-886b-d258c1c2278f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:37.592313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:24:37.630970 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7aa16-b0c7-45c4-886b-d258c1c2278f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:24:37.632348 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff7aa16-b0c7-45c4-886b-d258c1c2278f service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:24:37.650212 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 661/1325] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:24:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:24:39.825814 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd9eec-ef25-4de0-bbf7-c8371efc224c None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:39.826376 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd9eec-ef25-4de0-bbf7-c8371efc224c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:39.826376 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfd9eec-ef25-4de0-bbf7-c8371efc224c None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:39.827741 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 665/1326] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:24:39.835250 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd035fe-8ee0-4c03-97ce-398330c32317 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:39.835645 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd035fe-8ee0-4c03-97ce-398330c32317 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:39.835830 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd035fe-8ee0-4c03-97ce-398330c32317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:39.900522 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ddd035fe-8ee0-4c03-97ce-398330c32317 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:24:39.901719 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ddd035fe-8ee0-4c03-97ce-398330c32317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:40.002957 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 662/1327] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:24:40.283500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd9eec-ef25-4de0-bbf7-c8371efc224c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:24:40.285076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bbfd9eec-ef25-4de0-bbf7-c8371efc224c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:40.346396 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:24:40.389817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfd9eec-ef25-4de0-bbf7-c8371efc224c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:24:40.463484 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f735d70-e401-4a6f-80b6-a4c55b3a918e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:24:40.464095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f735d70-e401-4a6f-80b6-a4c55b3a918e service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:40.464373 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f735d70-e401-4a6f-80b6-a4c55b3a918e service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:40.464644 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f735d70-e401-4a6f-80b6-a4c55b3a918e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:40.466219 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4f735d70-e401-4a6f-80b6-a4c55b3a918e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:40.571896 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:24:40.574166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f735d70-e401-4a6f-80b6-a4c55b3a918e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:24:40.575641 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f735d70-e401-4a6f-80b6-a4c55b3a918e service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:24:40.593838 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 666/1328] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:24:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:24:42.544549 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d858e8-5774-4f17-8cc4-29e813deacd5 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:42.545087 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d858e8-5774-4f17-8cc4-29e813deacd5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:42.545087 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d858e8-5774-4f17-8cc4-29e813deacd5 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:42.546443 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 663/1329] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:24:42.551847 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b1798-8248-4fdd-a131-7ae9fc8982eb None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:42.552606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b1798-8248-4fdd-a131-7ae9fc8982eb None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:42.553132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b1798-8248-4fdd-a131-7ae9fc8982eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:42.637228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6c6b1798-8248-4fdd-a131-7ae9fc8982eb None None] MFA Rules not processed for user `604723d5b48f4e869bccbe56651aa80b`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:24:42.638467 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6c6b1798-8248-4fdd-a131-7ae9fc8982eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:42.743274 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 667/1330] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:24:42 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:24:43.023457 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d858e8-5774-4f17-8cc4-29e813deacd5 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:24:43.024968 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-44d858e8-5774-4f17-8cc4-29e813deacd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:43.055983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:24:43.126087 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d858e8-5774-4f17-8cc4-29e813deacd5 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:24:43.203049 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22134a97-a826-484b-998c-d27f002790d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:24:43.203675 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22134a97-a826-484b-998c-d27f002790d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:43.203954 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22134a97-a826-484b-998c-d27f002790d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:43.204229 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22134a97-a826-484b-998c-d27f002790d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:43.205841 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-22134a97-a826-484b-998c-d27f002790d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:43.289444 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:24:43.322461 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22134a97-a826-484b-998c-d27f002790d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:24:43.323941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22134a97-a826-484b-998c-d27f002790d8 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:24:43.342727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 664/1331] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:24:43 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:24:45.638498 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6ced7-629f-4690-a785-e77c6a36bfaf None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:45.639089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6ced7-629f-4690-a785-e77c6a36bfaf None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:45.639089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6ced7-629f-4690-a785-e77c6a36bfaf None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:45.640464 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 668/1332] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:24:45.645945 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d462860-c99c-4715-bdee-7c1d89f04925 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:45.646340 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d462860-c99c-4715-bdee-7c1d89f04925 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:45.646488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d462860-c99c-4715-bdee-7c1d89f04925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:45.711199 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2d462860-c99c-4715-bdee-7c1d89f04925 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:24:45.712381 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2d462860-c99c-4715-bdee-7c1d89f04925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:45.804534 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 665/1333] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:24:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:24:46.082888 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e6ced7-629f-4690-a785-e77c6a36bfaf None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:24:46.084428 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-95e6ced7-629f-4690-a785-e77c6a36bfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:46.158938 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:24:46.192282 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e6ced7-629f-4690-a785-e77c6a36bfaf None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:24:46.263503 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844eeeb-40de-4b06-b9a1-9942291710be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:24:46.264104 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844eeeb-40de-4b06-b9a1-9942291710be service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:46.264379 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844eeeb-40de-4b06-b9a1-9942291710be service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:46.264649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844eeeb-40de-4b06-b9a1-9942291710be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:46.266218 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a844eeeb-40de-4b06-b9a1-9942291710be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:46.371562 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844eeeb-40de-4b06-b9a1-9942291710be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:24:46.372986 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844eeeb-40de-4b06-b9a1-9942291710be service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:24:46.382762 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:24:46.390847 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 669/1334] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:24:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:24:48.334299 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605bece-7c43-4849-968c-d05451ece26a None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:48.335312 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605bece-7c43-4849-968c-d05451ece26a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:48.335312 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605bece-7c43-4849-968c-d05451ece26a None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:48.336637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 666/1335] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:24:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:24:48.341930 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a460f61-f65e-4579-9311-baa5d4469a09 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:48.342358 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a460f61-f65e-4579-9311-baa5d4469a09 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:48.342483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a460f61-f65e-4579-9311-baa5d4469a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:48.407321 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7a460f61-f65e-4579-9311-baa5d4469a09 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:24:48.408529 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7a460f61-f65e-4579-9311-baa5d4469a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:48.504946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 670/1336] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:24:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:24:48.787464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605bece-7c43-4849-968c-d05451ece26a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:24:48.789489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f605bece-7c43-4849-968c-d05451ece26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:48.828796 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:24:48.891778 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605bece-7c43-4849-968c-d05451ece26a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:24:48.964000 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862aced8-3670-4838-ba04-a1b9ac06f739 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:24:48.964604 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862aced8-3670-4838-ba04-a1b9ac06f739 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:48.964885 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862aced8-3670-4838-ba04-a1b9ac06f739 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:48.965156 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862aced8-3670-4838-ba04-a1b9ac06f739 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:48.966762 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-862aced8-3670-4838-ba04-a1b9ac06f739 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:49.047230 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:24:49.073465 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862aced8-3670-4838-ba04-a1b9ac06f739 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:24:49.075032 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862aced8-3670-4838-ba04-a1b9ac06f739 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:24:49.092918 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 667/1337] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:24:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:24:51.612910 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ab427-0e04-4cb1-882d-f36a2e9850d1 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:51.613503 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ab427-0e04-4cb1-882d-f36a2e9850d1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:51.613503 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6ab427-0e04-4cb1-882d-f36a2e9850d1 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:51.614954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 671/1338] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:24:51.620513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4dccdc-663f-4050-9997-74a61b517eee None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:51.620980 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4dccdc-663f-4050-9997-74a61b517eee None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:51.621084 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4dccdc-663f-4050-9997-74a61b517eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:51.687296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-7d4dccdc-663f-4050-9997-74a61b517eee None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:24:51.688500 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7d4dccdc-663f-4050-9997-74a61b517eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:51.782398 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 668/1339] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:24:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:24:52.074072 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ab427-0e04-4cb1-882d-f36a2e9850d1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:24:52.075663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9c6ab427-0e04-4cb1-882d-f36a2e9850d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:52.149786 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:24:52.176241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6ab427-0e04-4cb1-882d-f36a2e9850d1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:24:52.247116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c667ca8-3e4b-49c1-9a45-d7a5c919ae57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:24:52.247717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c667ca8-3e4b-49c1-9a45-d7a5c919ae57 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:52.247994 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c667ca8-3e4b-49c1-9a45-d7a5c919ae57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:52.248264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c667ca8-3e4b-49c1-9a45-d7a5c919ae57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:52.249842 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7c667ca8-3e4b-49c1-9a45-d7a5c919ae57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:52.356748 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c667ca8-3e4b-49c1-9a45-d7a5c919ae57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:24:52.358169 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c667ca8-3e4b-49c1-9a45-d7a5c919ae57 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:24:52.375635 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 672/1340] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:24:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:24:56.389275 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff4099-44e3-469c-a7d7-5b3c1c42205c None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:56.389813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff4099-44e3-469c-a7d7-5b3c1c42205c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:56.389813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff4099-44e3-469c-a7d7-5b3c1c42205c None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:56.391357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 669/1341] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:24:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:24:56.396311 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91421e-cfb8-4fe2-8fee-23c9e7101346 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:56.397304 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91421e-cfb8-4fe2-8fee-23c9e7101346 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:56.397463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91421e-cfb8-4fe2-8fee-23c9e7101346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:56.402924 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:24:56.467330 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-cc91421e-cfb8-4fe2-8fee-23c9e7101346 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:24:56.469544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cc91421e-cfb8-4fe2-8fee-23c9e7101346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:56.567088 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 673/1342] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:24:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:24:56.854366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff4099-44e3-469c-a7d7-5b3c1c42205c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:24:56.855991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b0ff4099-44e3-469c-a7d7-5b3c1c42205c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:56.905275 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:24:56.960613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff4099-44e3-469c-a7d7-5b3c1c42205c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:24:57.032068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b295915-a56e-4994-b08b-eb1461e05303 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:24:57.032679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b295915-a56e-4994-b08b-eb1461e05303 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:24:57.032955 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b295915-a56e-4994-b08b-eb1461e05303 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:24:57.033225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b295915-a56e-4994-b08b-eb1461e05303 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:24:57.035096 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4b295915-a56e-4994-b08b-eb1461e05303 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:24:57.126327 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:24:57.143586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b295915-a56e-4994-b08b-eb1461e05303 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:24:57.145025 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b295915-a56e-4994-b08b-eb1461e05303 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:24:57.164205 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 670/1343] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:24:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:25:00.046804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38109188-2637-454b-af49-8926f3d80405 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:00.047331 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38109188-2637-454b-af49-8926f3d80405 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:00.047331 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38109188-2637-454b-af49-8926f3d80405 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:00.048710 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 674/1344] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:25:00.054467 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103ff37-2ef0-42af-9dfe-a64fb71a137f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:00.054928 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103ff37-2ef0-42af-9dfe-a64fb71a137f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:00.055034 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103ff37-2ef0-42af-9dfe-a64fb71a137f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:00.119910 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8103ff37-2ef0-42af-9dfe-a64fb71a137f None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:25:00.121111 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8103ff37-2ef0-42af-9dfe-a64fb71a137f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:00.216778 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 671/1345] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:25:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:25:00.499853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38109188-2637-454b-af49-8926f3d80405 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:25:00.501404 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-38109188-2637-454b-af49-8926f3d80405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:00.590879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:00.609253 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38109188-2637-454b-af49-8926f3d80405 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:25:00.683963 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d086f7d-782b-46b8-90d0-4ec8034f5f55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:25:00.684546 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d086f7d-782b-46b8-90d0-4ec8034f5f55 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:00.684820 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d086f7d-782b-46b8-90d0-4ec8034f5f55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:00.685086 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d086f7d-782b-46b8-90d0-4ec8034f5f55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:00.686732 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5d086f7d-782b-46b8-90d0-4ec8034f5f55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:00.800361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d086f7d-782b-46b8-90d0-4ec8034f5f55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:25:00.802676 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d086f7d-782b-46b8-90d0-4ec8034f5f55 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:25:00.824171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 675/1346] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:25:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:25:04.973132 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a21ef-7ae8-4207-bdd6-948e1c656beb None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:04.973630 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a21ef-7ae8-4207-bdd6-948e1c656beb None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:04.973630 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a21ef-7ae8-4207-bdd6-948e1c656beb None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:04.975756 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 672/1347] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:25:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:25:04.980788 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6133b1f-5769-4807-8fb0-6ec9d882efcf None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:04.981243 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6133b1f-5769-4807-8fb0-6ec9d882efcf None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:04.981346 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6133b1f-5769-4807-8fb0-6ec9d882efcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:04.997610 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:05.052036 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c6133b1f-5769-4807-8fb0-6ec9d882efcf None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:25:05.053358 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c6133b1f-5769-4807-8fb0-6ec9d882efcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:05.155463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 676/1348] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:25:05.440907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a21ef-7ae8-4207-bdd6-948e1c656beb None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:25:05.442440 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3f7a21ef-7ae8-4207-bdd6-948e1c656beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:05.499408 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:05.545062 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a21ef-7ae8-4207-bdd6-948e1c656beb None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:25:05.616881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c971f4c-ed4e-4630-9d7f-fa811bbbb66d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:25:05.617522 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c971f4c-ed4e-4630-9d7f-fa811bbbb66d service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:05.617800 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c971f4c-ed4e-4630-9d7f-fa811bbbb66d service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:05.618178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c971f4c-ed4e-4630-9d7f-fa811bbbb66d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:05.619863 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5c971f4c-ed4e-4630-9d7f-fa811bbbb66d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:05.723012 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:05.727181 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c971f4c-ed4e-4630-9d7f-fa811bbbb66d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:25:05.728635 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c971f4c-ed4e-4630-9d7f-fa811bbbb66d service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:25:05.746502 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 673/1349] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:25:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:25:10.618682 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc490f6-c5f0-4b18-b981-5645d11d6a44 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:10.619211 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc490f6-c5f0-4b18-b981-5645d11d6a44 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:10.619504 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc490f6-c5f0-4b18-b981-5645d11d6a44 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:10.621254 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 677/1350] 10.222.0.57 () {54 vars in 921 bytes} [Fri Sep 2 09:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:25:10.626513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117569ee-6dd4-4641-b9be-8e22cb3fa7ab None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:10.627330 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117569ee-6dd4-4641-b9be-8e22cb3fa7ab None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:10.627774 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117569ee-6dd4-4641-b9be-8e22cb3fa7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:10.803734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-117569ee-6dd4-4641-b9be-8e22cb3fa7ab None None] MFA Rules not processed for user `5177fa4d168c4c219e09c7178bbbc790`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:25:10.804946 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-117569ee-6dd4-4641-b9be-8e22cb3fa7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:10.940857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 674/1351] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Sep 2 09:25:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:25:15.120546 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6038e-5ebe-4288-925c-77325a7986d9 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:15.121050 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6038e-5ebe-4288-925c-77325a7986d9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:15.121462 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6038e-5ebe-4288-925c-77325a7986d9 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:15.123284 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 678/1352] 10.222.0.57 () {54 vars in 921 bytes} [Fri Sep 2 09:25:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:25:15.128565 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6fb9a-e75b-446a-9a92-1c7ca0e57855 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:15.129011 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6fb9a-e75b-446a-9a92-1c7ca0e57855 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:15.129096 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6fb9a-e75b-446a-9a92-1c7ca0e57855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:15.153574 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=59 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:15.186876 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:15.212058 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0ca6fb9a-e75b-446a-9a92-1c7ca0e57855 None None] MFA Rules not processed for user `5177fa4d168c4c219e09c7178bbbc790`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:25:15.213271 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0ca6fb9a-e75b-446a-9a92-1c7ca0e57855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:15.340673 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 675/1353] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Sep 2 09:25:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:25:16.539820 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97baef8d-0d58-467c-9104-8fa492ab05fe None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:16.539820 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97baef8d-0d58-467c-9104-8fa492ab05fe None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:16.539820 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97baef8d-0d58-467c-9104-8fa492ab05fe None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:16.541295 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 679/1354] 10.222.0.57 () {54 vars in 921 bytes} [Fri Sep 2 09:25:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:25:16.547207 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6dd89f-c228-45d6-a61d-be64c394a300 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:16.547689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6dd89f-c228-45d6-a61d-be64c394a300 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:16.547905 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6dd89f-c228-45d6-a61d-be64c394a300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:16.625073 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ad6dd89f-c228-45d6-a61d-be64c394a300 None None] MFA Rules not processed for user `5177fa4d168c4c219e09c7178bbbc790`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:25:16.626282 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ad6dd89f-c228-45d6-a61d-be64c394a300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:16.668856 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:16.732664 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 676/1355] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Sep 2 09:25:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:25:20.027316 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48481969-780d-4042-8294-068bfb1e02d5 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:20.027864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48481969-780d-4042-8294-068bfb1e02d5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:20.027864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48481969-780d-4042-8294-068bfb1e02d5 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:20.029227 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 680/1356] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:25:20.034942 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec9774-19df-492c-9678-aeaaeb9c9c71 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:20.035382 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec9774-19df-492c-9678-aeaaeb9c9c71 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:20.035480 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec9774-19df-492c-9678-aeaaeb9c9c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:20.117640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-80ec9774-19df-492c-9678-aeaaeb9c9c71 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:25:20.119014 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-80ec9774-19df-492c-9678-aeaaeb9c9c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:20.249626 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 677/1357] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:25:20.557542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f05cf1-9ce9-429e-ac53-1e1868a74e79 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:20.558169 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f05cf1-9ce9-429e-ac53-1e1868a74e79 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:20.558663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f05cf1-9ce9-429e-ac53-1e1868a74e79 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:20.560684 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 681/1358] 10.222.0.57 () {54 vars in 962 bytes} [Fri Sep 2 09:25:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:25:20.568773 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d7c66-68bb-4b7e-84e3-0227b6bf9512 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:20.569320 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d7c66-68bb-4b7e-84e3-0227b6bf9512 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:20.571628 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d7c66-68bb-4b7e-84e3-0227b6bf9512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:20.589613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:20.654724 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6c6d7c66-68bb-4b7e-84e3-0227b6bf9512 None None] MFA Rules not processed for user `1da355c8be2d496bb08feb0fe74e04b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:25:20.667542 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6c6d7c66-68bb-4b7e-84e3-0227b6bf9512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:20.828244 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 678/1359] 10.222.0.57 () {58 vars in 1059 bytes} [Fri Sep 2 09:25:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:25:20.828518 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f05cf1-9ce9-429e-ac53-1e1868a74e79 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:25:20.828518 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-01f05cf1-9ce9-429e-ac53-1e1868a74e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:20.966199 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f05cf1-9ce9-429e-ac53-1e1868a74e79 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:25:21.052660 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:21.097114 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1771f3ac-89be-4bad-a51b-08934d139a23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:25:21.097790 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771f3ac-89be-4bad-a51b-08934d139a23 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:21.098222 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771f3ac-89be-4bad-a51b-08934d139a23 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:21.098563 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771f3ac-89be-4bad-a51b-08934d139a23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:21.100257 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1771f3ac-89be-4bad-a51b-08934d139a23 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:21.195184 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1771f3ac-89be-4bad-a51b-08934d139a23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:25:21.196799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1771f3ac-89be-4bad-a51b-08934d139a23 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:25:21.216046 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 682/1360] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:25:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:25:23.338871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763d336-46ae-44c3-8223-f50472cc42d3 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:23.339269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763d336-46ae-44c3-8223-f50472cc42d3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:23.339505 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763d336-46ae-44c3-8223-f50472cc42d3 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:23.344688 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 679/1361] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:25:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:25:23.346758 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede5c99-e0b0-4f04-8f21-a6df9141313e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:23.347149 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede5c99-e0b0-4f04-8f21-a6df9141313e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:23.347373 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede5c99-e0b0-4f04-8f21-a6df9141313e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:23.418540 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-eede5c99-e0b0-4f04-8f21-a6df9141313e None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:25:23.419682 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-eede5c99-e0b0-4f04-8f21-a6df9141313e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:23.459680 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:23.520216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 683/1362] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:25:23.808583 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9763d336-46ae-44c3-8223-f50472cc42d3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:25:23.810225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9763d336-46ae-44c3-8223-f50472cc42d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:23.889866 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:23.983785 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9763d336-46ae-44c3-8223-f50472cc42d3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:25:24.109853 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893a299-0c11-4a0c-b842-addd4d2f4389 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:25:24.110459 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893a299-0c11-4a0c-b842-addd4d2f4389 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:24.110756 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893a299-0c11-4a0c-b842-addd4d2f4389 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:24.111026 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893a299-0c11-4a0c-b842-addd4d2f4389 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:24.112648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d893a299-0c11-4a0c-b842-addd4d2f4389 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:24.136536 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:24.205930 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893a299-0c11-4a0c-b842-addd4d2f4389 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:25:24.207986 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893a299-0c11-4a0c-b842-addd4d2f4389 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:25:24.227304 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 680/1363] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:25:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:25:26.552163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce952a78-6cd6-4391-96cd-af3aa5550d81 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:26.566310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce952a78-6cd6-4391-96cd-af3aa5550d81 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:26.566310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce952a78-6cd6-4391-96cd-af3aa5550d81 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:26.566310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 684/1364] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:25:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:25:26.566623 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6a771-6d6c-4a69-b46f-6c2dcc9d7c79 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:26.566623 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6a771-6d6c-4a69-b46f-6c2dcc9d7c79 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:26.566623 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6a771-6d6c-4a69-b46f-6c2dcc9d7c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:26.642562 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-68a6a771-6d6c-4a69-b46f-6c2dcc9d7c79 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:25:26.643926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-68a6a771-6d6c-4a69-b46f-6c2dcc9d7c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:26.745868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:26.750765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 681/1365] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:25:27.040535 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb6e52-80ef-477e-8494-827b8791dccf None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:27.040818 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb6e52-80ef-477e-8494-827b8791dccf None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:27.041272 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb6e52-80ef-477e-8494-827b8791dccf None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:27.043061 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 685/1366] 10.222.0.57 () {54 vars in 962 bytes} [Fri Sep 2 09:25:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:25:27.049091 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e69e7-db50-4deb-9ae0-38cb19f3a3dc None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:27.049566 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e69e7-db50-4deb-9ae0-38cb19f3a3dc None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:27.049992 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e69e7-db50-4deb-9ae0-38cb19f3a3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:27.128103 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-7c1e69e7-db50-4deb-9ae0-38cb19f3a3dc None None] MFA Rules not processed for user `1da355c8be2d496bb08feb0fe74e04b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:25:27.130077 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7c1e69e7-db50-4deb-9ae0-38cb19f3a3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:27.253374 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 682/1367] 10.222.0.57 () {58 vars in 1059 bytes} [Fri Sep 2 09:25:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:25:27.261131 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb6e52-80ef-477e-8494-827b8791dccf None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:25:27.262777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7ddb6e52-80ef-477e-8494-827b8791dccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:27.414545 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb6e52-80ef-477e-8494-827b8791dccf None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:25:27.435752 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:27.538299 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960580c-1090-459d-a010-5d11d6b20bc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:25:27.539136 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960580c-1090-459d-a010-5d11d6b20bc1 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:27.539512 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960580c-1090-459d-a010-5d11d6b20bc1 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:27.539883 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960580c-1090-459d-a010-5d11d6b20bc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:27.541573 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d960580c-1090-459d-a010-5d11d6b20bc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:27.636903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d960580c-1090-459d-a010-5d11d6b20bc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:25:27.638493 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d960580c-1090-459d-a010-5d11d6b20bc1 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:25:27.656776 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 686/1368] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:25:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:25:28.294097 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d1e1f-5580-4d7f-b5ff-83915f50506b None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:28.295026 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d1e1f-5580-4d7f-b5ff-83915f50506b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:28.295475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d1e1f-5580-4d7f-b5ff-83915f50506b None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:28.297455 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 683/1369] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:25:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:25:28.303281 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff49b4-b9b5-4922-9ad2-e0aeee675ae5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:28.303736 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff49b4-b9b5-4922-9ad2-e0aeee675ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:28.304167 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff49b4-b9b5-4922-9ad2-e0aeee675ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:28.340274 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:28.382874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d6ff49b4-b9b5-4922-9ad2-e0aeee675ae5 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:25:28.384114 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d6ff49b4-b9b5-4922-9ad2-e0aeee675ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:28.495680 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 687/1370] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:25:29.072700 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d4907-b96c-4a15-aaba-9a0893636fba None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:29.073824 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d4907-b96c-4a15-aaba-9a0893636fba None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:29.073824 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4d4907-b96c-4a15-aaba-9a0893636fba None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:29.074756 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 684/1371] 10.222.0.57 () {54 vars in 964 bytes} [Fri Sep 2 09:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:25:29.079737 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb25c92-47c7-473c-a9cf-eb9640f64642 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:29.080226 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb25c92-47c7-473c-a9cf-eb9640f64642 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:29.080480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb25c92-47c7-473c-a9cf-eb9640f64642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:29.158541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-deb25c92-47c7-473c-a9cf-eb9640f64642 None None] MFA Rules not processed for user `b6a1f44f6fe94c09b696867871fd5b47`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:25:29.159780 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-deb25c92-47c7-473c-a9cf-eb9640f64642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:29.208229 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:29.242464 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 688/1372] 10.222.0.57 () {58 vars in 1061 bytes} [Fri Sep 2 09:25:29 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:25:29.250103 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d4907-b96c-4a15-aaba-9a0893636fba None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:25:29.251636 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1a4d4907-b96c-4a15-aaba-9a0893636fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:29.303992 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:29.362169 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4d4907-b96c-4a15-aaba-9a0893636fba None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:25:29.437724 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72400cc-2621-4284-b0f7-6169a4efb051 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:25:29.438401 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72400cc-2621-4284-b0f7-6169a4efb051 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:29.438810 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72400cc-2621-4284-b0f7-6169a4efb051 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:29.439205 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72400cc-2621-4284-b0f7-6169a4efb051 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:29.440974 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f72400cc-2621-4284-b0f7-6169a4efb051 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:29.552483 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:29.565964 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72400cc-2621-4284-b0f7-6169a4efb051 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:25:29.568666 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72400cc-2621-4284-b0f7-6169a4efb051 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:25:29.588572 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 685/1373] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:25:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:25:29.792594 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c268ce3-397d-4e0f-9a73-0384ad1c95d7 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:29.792975 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c268ce3-397d-4e0f-9a73-0384ad1c95d7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:29.793156 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c268ce3-397d-4e0f-9a73-0384ad1c95d7 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:29.794524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 689/1374] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:25:29.799801 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c1caf-3773-4025-b13e-9979c874758f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:29.800134 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c1caf-3773-4025-b13e-9979c874758f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:29.800291 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c1caf-3773-4025-b13e-9979c874758f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:29.893218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4d7c1caf-3773-4025-b13e-9979c874758f None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:25:29.895124 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4d7c1caf-3773-4025-b13e-9979c874758f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:30.000309 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 686/1375] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:25:30.290514 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c268ce3-397d-4e0f-9a73-0384ad1c95d7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:25:30.292068 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4c268ce3-397d-4e0f-9a73-0384ad1c95d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:30.440857 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c268ce3-397d-4e0f-9a73-0384ad1c95d7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:25:30.494340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:30.556685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1efc4d-beeb-4fa4-8ed7-0d8f6276c062 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:25:30.557226 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1efc4d-beeb-4fa4-8ed7-0d8f6276c062 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:30.557506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1efc4d-beeb-4fa4-8ed7-0d8f6276c062 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:30.557779 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1efc4d-beeb-4fa4-8ed7-0d8f6276c062 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:30.559413 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ef1efc4d-beeb-4fa4-8ed7-0d8f6276c062 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:30.650053 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1efc4d-beeb-4fa4-8ed7-0d8f6276c062 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:25:30.651677 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1efc4d-beeb-4fa4-8ed7-0d8f6276c062 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:25:30.670529 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 690/1376] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:25:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:25:32.729798 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e2aec-c582-45ba-aec1-0ad674b2adb5 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:32.730276 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e2aec-c582-45ba-aec1-0ad674b2adb5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:32.730380 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e2aec-c582-45ba-aec1-0ad674b2adb5 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:32.731730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 687/1377] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:25:32.737085 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245752c-6e24-4600-8778-67f13e2c0b33 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:32.737839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245752c-6e24-4600-8778-67f13e2c0b33 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:32.738815 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245752c-6e24-4600-8778-67f13e2c0b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:32.819231 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9245752c-6e24-4600-8778-67f13e2c0b33 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:25:32.820390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9245752c-6e24-4600-8778-67f13e2c0b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:32.830579 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:32.921378 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 691/1378] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:25:33.230757 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e2aec-c582-45ba-aec1-0ad674b2adb5 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:25:33.230757 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-550e2aec-c582-45ba-aec1-0ad674b2adb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:33.311620 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:33.403632 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e2aec-c582-45ba-aec1-0ad674b2adb5 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:25:33.528750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f347f6-b2b6-47dc-ab2b-98fadc2fd91c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:25:33.529755 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f347f6-b2b6-47dc-ab2b-98fadc2fd91c service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:33.530379 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f347f6-b2b6-47dc-ab2b-98fadc2fd91c service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:33.530902 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f347f6-b2b6-47dc-ab2b-98fadc2fd91c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:33.533049 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-00f347f6-b2b6-47dc-ab2b-98fadc2fd91c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:33.554659 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:33.635141 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f347f6-b2b6-47dc-ab2b-98fadc2fd91c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:25:33.636745 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f347f6-b2b6-47dc-ab2b-98fadc2fd91c service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:25:33.656375 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 688/1379] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:25:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:25:35.738200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c50b1-0b1d-443a-98eb-a0b316de9a94 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:35.738792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c50b1-0b1d-443a-98eb-a0b316de9a94 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:35.738881 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c50b1-0b1d-443a-98eb-a0b316de9a94 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:35.740432 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 692/1380] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:25:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:25:35.745613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b239a25-4585-4761-8ce8-3d122b5a4e34 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:35.746038 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b239a25-4585-4761-8ce8-3d122b5a4e34 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:35.746138 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b239a25-4585-4761-8ce8-3d122b5a4e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:35.815241 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1b239a25-4585-4761-8ce8-3d122b5a4e34 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:25:35.816933 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1b239a25-4585-4761-8ce8-3d122b5a4e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:35.922677 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:35.929477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 689/1381] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:25:36.220158 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c50b1-0b1d-443a-98eb-a0b316de9a94 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:25:36.221666 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-342c50b1-0b1d-443a-98eb-a0b316de9a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:36.370087 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:36.382275 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c50b1-0b1d-443a-98eb-a0b316de9a94 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:25:36.501404 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41265a47-79f6-43c4-b74a-9543df05c854 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:25:36.502022 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41265a47-79f6-43c4-b74a-9543df05c854 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:36.502304 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41265a47-79f6-43c4-b74a-9543df05c854 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:36.502578 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41265a47-79f6-43c4-b74a-9543df05c854 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:36.504779 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-41265a47-79f6-43c4-b74a-9543df05c854 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:36.602029 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41265a47-79f6-43c4-b74a-9543df05c854 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:25:36.603771 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41265a47-79f6-43c4-b74a-9543df05c854 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:25:36.616040 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:36.625221 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 693/1382] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:25:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:25:38.835127 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3389de7-0491-4070-ba05-1ce200015eb8 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:38.835655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3389de7-0491-4070-ba05-1ce200015eb8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:38.835655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3389de7-0491-4070-ba05-1ce200015eb8 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:38.837016 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 690/1383] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:25:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:25:38.842468 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9241cfb-06a9-417a-b521-56d8d3cbef0e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:38.842958 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9241cfb-06a9-417a-b521-56d8d3cbef0e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:38.843276 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9241cfb-06a9-417a-b521-56d8d3cbef0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:38.912879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a9241cfb-06a9-417a-b521-56d8d3cbef0e None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:25:38.914092 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a9241cfb-06a9-417a-b521-56d8d3cbef0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:39.012174 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 694/1384] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:25:39.303268 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3389de7-0491-4070-ba05-1ce200015eb8 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:25:39.304860 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d3389de7-0491-4070-ba05-1ce200015eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:39.458858 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3389de7-0491-4070-ba05-1ce200015eb8 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:25:39.527206 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:39.573564 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076ab25-b92e-42cb-befb-0bbd64ae25b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:25:39.574123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076ab25-b92e-42cb-befb-0bbd64ae25b6 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:39.574471 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076ab25-b92e-42cb-befb-0bbd64ae25b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:39.574770 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076ab25-b92e-42cb-befb-0bbd64ae25b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:39.576386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5076ab25-b92e-42cb-befb-0bbd64ae25b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:39.684553 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076ab25-b92e-42cb-befb-0bbd64ae25b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:25:39.686260 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076ab25-b92e-42cb-befb-0bbd64ae25b6 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:25:39.705354 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 691/1385] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:25:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:25:41.603518 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59de285-1f43-4c29-8b0a-d00d00b54711 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:41.604065 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59de285-1f43-4c29-8b0a-d00d00b54711 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:41.604065 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59de285-1f43-4c29-8b0a-d00d00b54711 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:41.605422 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 695/1386] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:25:41.610814 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab9c66-c245-4aff-a42a-4066720779b3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:41.611670 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab9c66-c245-4aff-a42a-4066720779b3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:41.611670 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab9c66-c245-4aff-a42a-4066720779b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:41.683856 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4bab9c66-c245-4aff-a42a-4066720779b3 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:25:41.685403 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4bab9c66-c245-4aff-a42a-4066720779b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:41.726613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:41.793855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 692/1387] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:25:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:25:42.097644 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59de285-1f43-4c29-8b0a-d00d00b54711 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:25:42.099127 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b59de285-1f43-4c29-8b0a-d00d00b54711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:42.178663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:42.257453 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59de285-1f43-4c29-8b0a-d00d00b54711 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:25:42.381613 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536580e-e56b-4d73-9103-e2b8c08571c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:25:42.382456 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536580e-e56b-4d73-9103-e2b8c08571c4 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:42.382834 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536580e-e56b-4d73-9103-e2b8c08571c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:42.383184 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536580e-e56b-4d73-9103-e2b8c08571c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:42.385278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1536580e-e56b-4d73-9103-e2b8c08571c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:42.431598 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:42.483660 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536580e-e56b-4d73-9103-e2b8c08571c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:25:42.485268 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536580e-e56b-4d73-9103-e2b8c08571c4 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:25:42.506578 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 696/1388] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:25:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:25:44.545033 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0a8e5-3e78-47e7-a663-847eede5fd2a None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:44.545561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0a8e5-3e78-47e7-a663-847eede5fd2a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:44.545561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc0a8e5-3e78-47e7-a663-847eede5fd2a None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:44.546951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 693/1389] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:25:44.552518 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd25b47-9dd6-4791-b01a-c3063b1a495b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:44.552910 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd25b47-9dd6-4791-b01a-c3063b1a495b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:44.553074 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd25b47-9dd6-4791-b01a-c3063b1a495b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:44.619431 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5bd25b47-9dd6-4791-b01a-c3063b1a495b None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:25:44.620650 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5bd25b47-9dd6-4791-b01a-c3063b1a495b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:44.716905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 697/1390] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:25:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:25:44.822807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b97a24-43cd-4e66-b53d-37a41bb74812 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:44.823098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b97a24-43cd-4e66-b53d-37a41bb74812 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:44.823370 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b97a24-43cd-4e66-b53d-37a41bb74812 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:44.824855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 694/1391] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:25:44.829803 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a834417-ecb1-4134-98a2-6a4e863cb08e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:44.830098 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a834417-ecb1-4134-98a2-6a4e863cb08e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:44.830377 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a834417-ecb1-4134-98a2-6a4e863cb08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:44.847296 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:44.898138 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9a834417-ecb1-4134-98a2-6a4e863cb08e None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:25:44.899391 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9a834417-ecb1-4134-98a2-6a4e863cb08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:44.997854 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 698/1392] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:25:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:25:45.001895 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b97a24-43cd-4e66-b53d-37a41bb74812 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:25:45.003807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-63b97a24-43cd-4e66-b53d-37a41bb74812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:45.147202 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b97a24-43cd-4e66-b53d-37a41bb74812 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:25:45.163929 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:45.262207 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd888b33-7390-4e32-8bba-299eb4f3ff66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:25:45.262747 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd888b33-7390-4e32-8bba-299eb4f3ff66 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:45.263023 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd888b33-7390-4e32-8bba-299eb4f3ff66 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:45.263291 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd888b33-7390-4e32-8bba-299eb4f3ff66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:45.264897 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fd888b33-7390-4e32-8bba-299eb4f3ff66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:45.303012 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7cc2d-64e3-46c5-987b-0b45dbcbb84b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:45.303303 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7cc2d-64e3-46c5-987b-0b45dbcbb84b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:45.303579 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7cc2d-64e3-46c5-987b-0b45dbcbb84b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:45.304983 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 699/1393] 10.222.0.57 () {54 vars in 961 bytes} [Fri Sep 2 09:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:25:45.309641 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87a2ee-5f27-488f-b985-205821697784 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:45.309919 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87a2ee-5f27-488f-b985-205821697784 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:45.310190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87a2ee-5f27-488f-b985-205821697784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:45.355449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd888b33-7390-4e32-8bba-299eb4f3ff66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:25:45.357046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd888b33-7390-4e32-8bba-299eb4f3ff66 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:25:45.375525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 695/1394] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:25:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:25:45.392857 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6d87a2ee-5f27-488f-b985-205821697784 None None] MFA Rules not processed for user `31b555bb94ce44ce918cbf2a012576a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:25:45.394105 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6d87a2ee-5f27-488f-b985-205821697784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:45.428586 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:45.473526 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 700/1395] 10.222.0.57 () {58 vars in 1058 bytes} [Fri Sep 2 09:25:45 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:25:45.481096 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd888b33-7390-4e32-8bba-299eb4f3ff66 service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:25:45.484849 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fd888b33-7390-4e32-8bba-299eb4f3ff66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:45.505187 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:45.558907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd888b33-7390-4e32-8bba-299eb4f3ff66 service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:25:45.642395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65544c25-8b8e-442f-ba08-501c6a2c5d50 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b555bb94ce44ce918cbf2a012576a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:25:45.643074 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65544c25-8b8e-442f-ba08-501c6a2c5d50 service glance] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:45.643359 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65544c25-8b8e-442f-ba08-501c6a2c5d50 service glance] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:45.643634 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65544c25-8b8e-442f-ba08-501c6a2c5d50 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:45.645366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-65544c25-8b8e-442f-ba08-501c6a2c5d50 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:45.746834 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:45.759586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65544c25-8b8e-442f-ba08-501c6a2c5d50 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:25:45.764428 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65544c25-8b8e-442f-ba08-501c6a2c5d50 service glance] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:25:45.778831 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 696/1396] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:25:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:25:46.782469 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9c918-61ab-486a-887c-cf8fb2a395cc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:46.782469 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9c918-61ab-486a-887c-cf8fb2a395cc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:46.782469 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9c918-61ab-486a-887c-cf8fb2a395cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:46.846018 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1cf9c918-61ab-486a-887c-cf8fb2a395cc None None] MFA Rules not processed for user `5233689a53364258b86ad2fa012d8db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:25:46.847445 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1cf9c918-61ab-486a-887c-cf8fb2a395cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:46.936483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 701/1397] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Sep 2 09:25:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:25:46.979707 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d6c46-89c1-42ae-bc3c-1829a6153e5e None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:46.980225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d6c46-89c1-42ae-bc3c-1829a6153e5e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:46.980225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d6c46-89c1-42ae-bc3c-1829a6153e5e None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:46.981805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 697/1398] 10.222.0.57 () {54 vars in 943 bytes} [Fri Sep 2 09:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:25:46.988479 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f48d5-d7b5-4dda-852c-9509dab34afc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:46.988924 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f48d5-d7b5-4dda-852c-9509dab34afc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:46.989118 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f48d5-d7b5-4dda-852c-9509dab34afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:47.047288 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:47.067329 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-956f48d5-d7b5-4dda-852c-9509dab34afc None None] MFA Rules not processed for user `30fe233a7d4c4fbea6ede233bd9ff8c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:25:47.068528 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-956f48d5-d7b5-4dda-852c-9509dab34afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:47.176123 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 702/1399] 10.222.0.57 () {58 vars in 1040 bytes} [Fri Sep 2 09:25:46 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:25:47.185472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d6c46-89c1-42ae-bc3c-1829a6153e5e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:25:47.186994 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2a3d6c46-89c1-42ae-bc3c-1829a6153e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:47.321499 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d6c46-89c1-42ae-bc3c-1829a6153e5e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:25:47.385878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:47.416728 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c09ac-2a5c-42fa-b9cc-2578570179ec None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:47.417014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c09ac-2a5c-42fa-b9cc-2578570179ec None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:47.417363 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c09ac-2a5c-42fa-b9cc-2578570179ec None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:47.418901 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 703/1400] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:25:47.423964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ef2e8-f001-460a-8afd-de072a3c628b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:47.424245 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ef2e8-f001-460a-8afd-de072a3c628b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:47.425157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ef2e8-f001-460a-8afd-de072a3c628b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:47.426976 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12a0a2-fb46-447f-92eb-572dea869602 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30fe233a7d4c4fbea6ede233bd9ff8c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:25:47.427586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12a0a2-fb46-447f-92eb-572dea869602 service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:47.427864 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12a0a2-fb46-447f-92eb-572dea869602 service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:47.428133 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12a0a2-fb46-447f-92eb-572dea869602 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:47.429782 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ec12a0a2-fb46-447f-92eb-572dea869602 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:47.490413 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e54ef2e8-f001-460a-8afd-de072a3c628b None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:25:47.491719 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e54ef2e8-f001-460a-8afd-de072a3c628b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:47.512410 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12a0a2-fb46-447f-92eb-572dea869602 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:25:47.514004 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12a0a2-fb46-447f-92eb-572dea869602 service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:25:47.522599 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 698/1401] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Sep 2 09:25:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 09:25:47.549791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:47.590907 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 704/1402] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:25:47.895157 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12a0a2-fb46-447f-92eb-572dea869602 service swift] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:25:47.896373 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ec12a0a2-fb46-447f-92eb-572dea869602 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:47.985952 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12a0a2-fb46-447f-92eb-572dea869602 service swift] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:25:48.003421 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:48.107247 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41f796f-a49d-4d0f-ad64-8b0cb5623100 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:25:48.107771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f796f-a49d-4d0f-ad64-8b0cb5623100 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:48.108046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f796f-a49d-4d0f-ad64-8b0cb5623100 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:48.108313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41f796f-a49d-4d0f-ad64-8b0cb5623100 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:48.109911 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c41f796f-a49d-4d0f-ad64-8b0cb5623100 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:48.210607 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41f796f-a49d-4d0f-ad64-8b0cb5623100 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:25:48.212222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41f796f-a49d-4d0f-ad64-8b0cb5623100 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:25:48.233703 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 699/1403] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:25:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:25:50.302594 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44303313-4574-471a-95ff-8ac3db2a16ff None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:50.303270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44303313-4574-471a-95ff-8ac3db2a16ff None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:50.303270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44303313-4574-471a-95ff-8ac3db2a16ff None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:50.304541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 705/1404] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:25:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:25:50.309794 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15466328-2565-4878-9007-e18fff713750 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:50.310333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15466328-2565-4878-9007-e18fff713750 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:50.310333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15466328-2565-4878-9007-e18fff713750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:50.338189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:50.389368 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-15466328-2565-4878-9007-e18fff713750 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:25:50.390594 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-15466328-2565-4878-9007-e18fff713750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:50.500755 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 700/1405] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:25:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:25:50.793131 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44303313-4574-471a-95ff-8ac3db2a16ff None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:25:50.794790 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-44303313-4574-471a-95ff-8ac3db2a16ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:50.973968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44303313-4574-471a-95ff-8ac3db2a16ff None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:25:51.020273 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:51.101185 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7666f287-6558-4727-9902-97e9b7501e21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:25:51.106287 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666f287-6558-4727-9902-97e9b7501e21 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:51.106287 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666f287-6558-4727-9902-97e9b7501e21 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:51.106287 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666f287-6558-4727-9902-97e9b7501e21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:51.106287 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7666f287-6558-4727-9902-97e9b7501e21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:51.206419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7666f287-6558-4727-9902-97e9b7501e21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:25:51.208070 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7666f287-6558-4727-9902-97e9b7501e21 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:25:51.226790 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 706/1406] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:25:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:25:53.266790 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac6bad-be25-4a39-9d3d-7154de68f865 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:53.267317 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac6bad-be25-4a39-9d3d-7154de68f865 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:53.267317 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac6bad-be25-4a39-9d3d-7154de68f865 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:53.269235 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 701/1407] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:25:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:25:53.274508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5f4d2-3b93-402d-a7ac-4bce39ef359e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:53.274983 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5f4d2-3b93-402d-a7ac-4bce39ef359e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:53.275078 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5f4d2-3b93-402d-a7ac-4bce39ef359e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:53.337811 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:53.347855 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8eb5f4d2-3b93-402d-a7ac-4bce39ef359e None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:25:53.349085 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8eb5f4d2-3b93-402d-a7ac-4bce39ef359e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:53.456704 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 707/1408] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:25:53.752897 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac6bad-be25-4a39-9d3d-7154de68f865 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:25:53.754524 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e8ac6bad-be25-4a39-9d3d-7154de68f865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:53.865456 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:53.928103 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac6bad-be25-4a39-9d3d-7154de68f865 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:25:54.053264 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca052f-3b06-4bcf-a98d-4f08ef5e807c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:25:54.053866 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca052f-3b06-4bcf-a98d-4f08ef5e807c service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:25:54.054153 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca052f-3b06-4bcf-a98d-4f08ef5e807c service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:25:54.054428 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca052f-3b06-4bcf-a98d-4f08ef5e807c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:25:54.056092 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-39ca052f-3b06-4bcf-a98d-4f08ef5e807c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:25:54.121969 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:25:54.151538 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ca052f-3b06-4bcf-a98d-4f08ef5e807c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:25:54.153189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ca052f-3b06-4bcf-a98d-4f08ef5e807c service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:25:54.177314 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 702/1409] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:25:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:26:03.454397 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=44 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:04.174613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=18 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:11.341741 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb41ea3-d8da-45c5-8619-a9b836b5d8f8 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:11.342266 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb41ea3-d8da-45c5-8619-a9b836b5d8f8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:11.342266 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb41ea3-d8da-45c5-8619-a9b836b5d8f8 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:11.343599 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 708/1410] 10.222.0.57 () {54 vars in 918 bytes} [Fri Sep 2 09:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:26:11.348871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830652c-49a1-4276-8433-f0c242ac4d7b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:11.349294 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830652c-49a1-4276-8433-f0c242ac4d7b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:11.349421 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830652c-49a1-4276-8433-f0c242ac4d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:11.423639 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5830652c-49a1-4276-8433-f0c242ac4d7b None None] MFA Rules not processed for user `5177fa4d168c4c219e09c7178bbbc790`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:26:11.424846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5830652c-49a1-4276-8433-f0c242ac4d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:11.529229 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 703/1411] 10.222.0.57 () {58 vars in 1015 bytes} [Fri Sep 2 09:26:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:26:11.543469 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f52bf5-526a-473c-95ef-f9ce6c42ad3d None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:11.543758 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f52bf5-526a-473c-95ef-f9ce6c42ad3d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:11.544037 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f52bf5-526a-473c-95ef-f9ce6c42ad3d None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:11.545407 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 709/1412] 10.222.0.57 () {54 vars in 918 bytes} [Fri Sep 2 09:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:26:11.550351 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32411c6-b13b-4c3f-a795-f7ff1727a592 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:11.550639 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32411c6-b13b-4c3f-a795-f7ff1727a592 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:11.550941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32411c6-b13b-4c3f-a795-f7ff1727a592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:11.621839 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:11.624804 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a32411c6-b13b-4c3f-a795-f7ff1727a592 None None] MFA Rules not processed for user `1da355c8be2d496bb08feb0fe74e04b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:26:11.626634 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a32411c6-b13b-4c3f-a795-f7ff1727a592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:11.739121 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 704/1413] 10.222.0.57 () {58 vars in 1015 bytes} [Fri Sep 2 09:26:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:26:14.103601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879f696-8eff-4bc9-98cc-6f5d50382f79 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:14.104117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879f696-8eff-4bc9-98cc-6f5d50382f79 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:14.104117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879f696-8eff-4bc9-98cc-6f5d50382f79 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:14.105446 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 710/1414] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:26:14.110596 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5c11c-4d51-4a0b-a55f-fbaa85013ae2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:14.111035 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5c11c-4d51-4a0b-a55f-fbaa85013ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:14.111171 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac5c11c-4d51-4a0b-a55f-fbaa85013ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:14.176423 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-fac5c11c-4d51-4a0b-a55f-fbaa85013ae2 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:26:14.177623 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fac5c11c-4d51-4a0b-a55f-fbaa85013ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:14.233778 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:14.275754 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 705/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:26:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:26:14.562609 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879f696-8eff-4bc9-98cc-6f5d50382f79 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:26:14.564266 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a879f696-8eff-4bc9-98cc-6f5d50382f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:14.706537 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a879f696-8eff-4bc9-98cc-6f5d50382f79 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:26:14.776958 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:14.817131 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea54b6-6d38-4271-9738-9e954a38626e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:26:14.817736 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea54b6-6d38-4271-9738-9e954a38626e service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:14.818015 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea54b6-6d38-4271-9738-9e954a38626e service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:14.818292 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea54b6-6d38-4271-9738-9e954a38626e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:14.819935 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ecea54b6-6d38-4271-9738-9e954a38626e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:14.912450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea54b6-6d38-4271-9738-9e954a38626e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:26:14.914045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecea54b6-6d38-4271-9738-9e954a38626e service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:26:14.931400 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 711/1416] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:26:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:26:15.815514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e934b1a-c701-4030-854d-a559c01b6289 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:15.816082 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e934b1a-c701-4030-854d-a559c01b6289 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:15.816082 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e934b1a-c701-4030-854d-a559c01b6289 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:15.817436 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 706/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 09:26:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:26:15.822430 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622253bb-f33e-41e2-b816-38a9d65e8a0d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:15.822849 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622253bb-f33e-41e2-b816-38a9d65e8a0d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:15.823003 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622253bb-f33e-41e2-b816-38a9d65e8a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:15.900320 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-622253bb-f33e-41e2-b816-38a9d65e8a0d None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:26:15.901564 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-622253bb-f33e-41e2-b816-38a9d65e8a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:15.927743 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:15.995578 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 712/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 09:26:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:26:16.145589 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e934b1a-c701-4030-854d-a559c01b6289 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:26:16.147124 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3e934b1a-c701-4030-854d-a559c01b6289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:16.291181 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e934b1a-c701-4030-854d-a559c01b6289 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:26:16.330674 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:16.401742 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd2a03b-9270-4ee2-9400-ee9ce115b42a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:26:16.402281 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2a03b-9270-4ee2-9400-ee9ce115b42a service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:16.402561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2a03b-9270-4ee2-9400-ee9ce115b42a service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:16.402876 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2a03b-9270-4ee2-9400-ee9ce115b42a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:16.404477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fbd2a03b-9270-4ee2-9400-ee9ce115b42a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:16.492350 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd2a03b-9270-4ee2-9400-ee9ce115b42a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:26:16.493950 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd2a03b-9270-4ee2-9400-ee9ce115b42a service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:26:16.511882 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 707/1419] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:26:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:26:17.396633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e9e2f4-28bf-468a-b414-72e1deef8410 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:17.397199 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e9e2f4-28bf-468a-b414-72e1deef8410 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:17.397199 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e9e2f4-28bf-468a-b414-72e1deef8410 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:17.398544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 713/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 09:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:26:17.403537 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983d977-0434-4a4e-9d3f-aa29ecf06f8d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:17.403933 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983d977-0434-4a4e-9d3f-aa29ecf06f8d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:17.404115 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983d977-0434-4a4e-9d3f-aa29ecf06f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:17.458157 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:17.481747 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d983d977-0434-4a4e-9d3f-aa29ecf06f8d None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:26:17.482997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d983d977-0434-4a4e-9d3f-aa29ecf06f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:17.579905 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 708/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 09:26:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:26:17.731694 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e9e2f4-28bf-468a-b414-72e1deef8410 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:26:17.733056 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e4e9e2f4-28bf-468a-b414-72e1deef8410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:17.879482 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e9e2f4-28bf-468a-b414-72e1deef8410 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:26:17.891684 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:17.991157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a6e87-0b71-4484-a8fb-f499e77502ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:26:17.991692 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a6e87-0b71-4484-a8fb-f499e77502ff service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:17.991972 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a6e87-0b71-4484-a8fb-f499e77502ff service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:17.992240 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a6e87-0b71-4484-a8fb-f499e77502ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:17.993839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-748a6e87-0b71-4484-a8fb-f499e77502ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:18.081705 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a6e87-0b71-4484-a8fb-f499e77502ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:26:18.083334 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a6e87-0b71-4484-a8fb-f499e77502ff service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:26:18.101508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 714/1422] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:26:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:26:18.960166 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2842f-6244-4367-a40e-a72527d108f8 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:18.960723 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2842f-6244-4367-a40e-a72527d108f8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:18.960723 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca2842f-6244-4367-a40e-a72527d108f8 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:18.962059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 709/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 09:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:26:18.966905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144b1d2-72bd-4d3a-8e27-a3367e36cf98 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:18.967339 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144b1d2-72bd-4d3a-8e27-a3367e36cf98 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:18.967491 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2144b1d2-72bd-4d3a-8e27-a3367e36cf98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:18.987617 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:19.045333 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2144b1d2-72bd-4d3a-8e27-a3367e36cf98 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:26:19.046566 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2144b1d2-72bd-4d3a-8e27-a3367e36cf98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:19.142443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 715/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 09:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:26:19.292030 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2842f-6244-4367-a40e-a72527d108f8 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:26:19.293537 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9ca2842f-6244-4367-a40e-a72527d108f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:19.402190 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:19.444880 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca2842f-6244-4367-a40e-a72527d108f8 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:26:19.553702 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7d480-1772-4feb-8768-10bcc097f446 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:26:19.554304 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7d480-1772-4feb-8768-10bcc097f446 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:19.554580 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7d480-1772-4feb-8768-10bcc097f446 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:19.554880 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7d480-1772-4feb-8768-10bcc097f446 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:19.556473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e7d7d480-1772-4feb-8768-10bcc097f446 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:19.630768 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:19.644625 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d7d480-1772-4feb-8768-10bcc097f446 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:26:19.646238 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d7d480-1772-4feb-8768-10bcc097f446 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:26:19.667852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 710/1425] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:26:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:26:20.525034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571587f-b1d4-4f94-b9a7-116da3032e90 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:20.525526 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571587f-b1d4-4f94-b9a7-116da3032e90 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:20.525630 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9571587f-b1d4-4f94-b9a7-116da3032e90 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:20.526998 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 716/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 09:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:26:20.532290 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6535a7c-915c-48eb-bff3-778ec142c661 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:20.532730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6535a7c-915c-48eb-bff3-778ec142c661 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:20.532837 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6535a7c-915c-48eb-bff3-778ec142c661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:20.606680 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b6535a7c-915c-48eb-bff3-778ec142c661 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:26:20.607900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b6535a7c-915c-48eb-bff3-778ec142c661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:20.701529 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 711/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 09:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:26:20.850272 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9571587f-b1d4-4f94-b9a7-116da3032e90 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:26:20.851824 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9571587f-b1d4-4f94-b9a7-116da3032e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:20.938839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:21.000124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9571587f-b1d4-4f94-b9a7-116da3032e90 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:26:21.114100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6318d4b-5123-43bc-9b51-455a25f1a3fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:26:21.114632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6318d4b-5123-43bc-9b51-455a25f1a3fc service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:21.114941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6318d4b-5123-43bc-9b51-455a25f1a3fc service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:21.115212 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6318d4b-5123-43bc-9b51-455a25f1a3fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:21.116806 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b6318d4b-5123-43bc-9b51-455a25f1a3fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:21.164853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:21.206668 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6318d4b-5123-43bc-9b51-455a25f1a3fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:26:21.208333 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6318d4b-5123-43bc-9b51-455a25f1a3fc service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:26:21.226564 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 717/1428] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:26:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:26:22.161156 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30901780-dfee-411a-bba1-79bce9fb9ed4 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:22.161674 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30901780-dfee-411a-bba1-79bce9fb9ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:22.161784 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30901780-dfee-411a-bba1-79bce9fb9ed4 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:22.163072 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 712/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 09:26:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:26:22.167901 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1e96a-b5b2-4666-ba06-72e9950dc4de None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:22.168313 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1e96a-b5b2-4666-ba06-72e9950dc4de None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:22.168474 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1e96a-b5b2-4666-ba06-72e9950dc4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:22.243346 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1ad1e96a-b5b2-4666-ba06-72e9950dc4de None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:26:22.244549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1ad1e96a-b5b2-4666-ba06-72e9950dc4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:22.338352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 718/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 09:26:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:26:22.487714 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30901780-dfee-411a-bba1-79bce9fb9ed4 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:26:22.489222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-30901780-dfee-411a-bba1-79bce9fb9ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:22.537424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:22.634616 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30901780-dfee-411a-bba1-79bce9fb9ed4 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:26:22.745784 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f530feaf-8dba-406a-a4dd-03cdce108b0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:26:22.746314 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530feaf-8dba-406a-a4dd-03cdce108b0c service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:22.746596 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530feaf-8dba-406a-a4dd-03cdce108b0c service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:22.746903 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f530feaf-8dba-406a-a4dd-03cdce108b0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:22.748512 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f530feaf-8dba-406a-a4dd-03cdce108b0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:22.755373 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:22.836992 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530feaf-8dba-406a-a4dd-03cdce108b0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:26:22.838736 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f530feaf-8dba-406a-a4dd-03cdce108b0c service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:26:22.857449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 713/1431] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:26:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:26:23.709178 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c36991-9f95-4111-84af-a4ad4a5359a0 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:23.709754 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c36991-9f95-4111-84af-a4ad4a5359a0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:23.709754 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c36991-9f95-4111-84af-a4ad4a5359a0 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:23.711097 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 719/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 09:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:26:23.716029 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a6f48-2cf2-4a5d-9c65-54ee77991c6a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:23.716489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a6f48-2cf2-4a5d-9c65-54ee77991c6a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:23.716599 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a6f48-2cf2-4a5d-9c65-54ee77991c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:23.791171 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1d7a6f48-2cf2-4a5d-9c65-54ee77991c6a None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:26:23.792408 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1d7a6f48-2cf2-4a5d-9c65-54ee77991c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:23.860208 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:23.891150 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 714/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 09:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:26:24.041673 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c36991-9f95-4111-84af-a4ad4a5359a0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:26:24.043140 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-89c36991-9f95-4111-84af-a4ad4a5359a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:24.061022 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:24.193859 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c36991-9f95-4111-84af-a4ad4a5359a0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:26:24.279679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:24.305063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b81545f-1b4f-4fa3-b855-2356dd7d938f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:26:24.305603 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81545f-1b4f-4fa3-b855-2356dd7d938f service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:24.305881 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81545f-1b4f-4fa3-b855-2356dd7d938f service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:24.306150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b81545f-1b4f-4fa3-b855-2356dd7d938f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:24.307846 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8b81545f-1b4f-4fa3-b855-2356dd7d938f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:24.397040 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b81545f-1b4f-4fa3-b855-2356dd7d938f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:26:24.398626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b81545f-1b4f-4fa3-b855-2356dd7d938f service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:26:24.416364 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 720/1434] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:26:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:26:25.334565 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380832b-817a-47f0-b4ce-7464bde98fd2 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:25.335156 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380832b-817a-47f0-b4ce-7464bde98fd2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:25.335156 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1380832b-817a-47f0-b4ce-7464bde98fd2 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:25.336504 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 715/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 09:26:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:26:25.341303 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003a009-07ae-42ce-81e5-7cdbb38e55d0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:25.341753 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003a009-07ae-42ce-81e5-7cdbb38e55d0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:25.341867 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003a009-07ae-42ce-81e5-7cdbb38e55d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:25.416603 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e003a009-07ae-42ce-81e5-7cdbb38e55d0 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:26:25.417806 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e003a009-07ae-42ce-81e5-7cdbb38e55d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:25.459083 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:25.513310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 721/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 09:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:26:25.663462 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1380832b-817a-47f0-b4ce-7464bde98fd2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:26:25.665003 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1380832b-817a-47f0-b4ce-7464bde98fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:25.808139 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1380832b-817a-47f0-b4ce-7464bde98fd2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:26:25.862127 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:25.921427 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c16f6-6e1e-428b-b654-3e74e4b90ec5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:26:25.921957 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c16f6-6e1e-428b-b654-3e74e4b90ec5 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:25.922234 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c16f6-6e1e-428b-b654-3e74e4b90ec5 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:25.922501 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c16f6-6e1e-428b-b654-3e74e4b90ec5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:25.924133 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c75c16f6-6e1e-428b-b654-3e74e4b90ec5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:26.011602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c16f6-6e1e-428b-b654-3e74e4b90ec5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:26:26.013174 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c16f6-6e1e-428b-b654-3e74e4b90ec5 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:26:26.031902 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 716/1437] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:26:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:26:26.896149 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bc4b2-02ed-49b2-b6f1-fa973dc48708 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:26.896668 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bc4b2-02ed-49b2-b6f1-fa973dc48708 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:26.896668 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bc4b2-02ed-49b2-b6f1-fa973dc48708 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:26.898049 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 722/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 09:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:26:26.902855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03d0d3-c56a-4233-9376-1f7e1f85f41a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:26.903302 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03d0d3-c56a-4233-9376-1f7e1f85f41a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:26.903410 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b03d0d3-c56a-4233-9376-1f7e1f85f41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:26.977129 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4b03d0d3-c56a-4233-9376-1f7e1f85f41a None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:26:26.978324 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4b03d0d3-c56a-4233-9376-1f7e1f85f41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:26.980340 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:27.074005 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 717/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 09:26:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:26:27.226387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bc4b2-02ed-49b2-b6f1-fa973dc48708 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:26:27.227869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ef1bc4b2-02ed-49b2-b6f1-fa973dc48708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:27.370796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bc4b2-02ed-49b2-b6f1-fa973dc48708 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:26:27.397218 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:27.481651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4e301-074a-4450-ad30-a76b9365d10c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:26:27.482186 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4e301-074a-4450-ad30-a76b9365d10c service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:27.482465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4e301-074a-4450-ad30-a76b9365d10c service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:27.482761 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4e301-074a-4450-ad30-a76b9365d10c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:27.484852 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bcb4e301-074a-4450-ad30-a76b9365d10c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:27.574015 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4e301-074a-4450-ad30-a76b9365d10c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:26:27.579935 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4e301-074a-4450-ad30-a76b9365d10c service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:26:27.594236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 723/1440] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:26:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:26:29.418175 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f336b0a-108e-441b-89ba-f0a27d37e298 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:29.418883 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f336b0a-108e-441b-89ba-f0a27d37e298 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:29.418883 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f336b0a-108e-441b-89ba-f0a27d37e298 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:29.420131 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 718/1441] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:26:29.425304 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8dbf6-c457-43e0-bdd5-fa34eadfa811 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:29.425709 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8dbf6-c457-43e0-bdd5-fa34eadfa811 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:29.425863 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f8dbf6-c457-43e0-bdd5-fa34eadfa811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:29.466600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:29.491677 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c3f8dbf6-c457-43e0-bdd5-fa34eadfa811 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:26:29.493150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c3f8dbf6-c457-43e0-bdd5-fa34eadfa811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:29.596197 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 724/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:26:29.880049 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f336b0a-108e-441b-89ba-f0a27d37e298 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:26:29.881586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0f336b0a-108e-441b-89ba-f0a27d37e298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:29.985471 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f336b0a-108e-441b-89ba-f0a27d37e298 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:26:30.016568 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:30.057857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcf6761-0c7c-405f-bc77-b40d6563133e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:26:30.058437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf6761-0c7c-405f-bc77-b40d6563133e service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:30.058735 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf6761-0c7c-405f-bc77-b40d6563133e service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:30.059039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcf6761-0c7c-405f-bc77-b40d6563133e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:30.060614 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9fcf6761-0c7c-405f-bc77-b40d6563133e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:30.167495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf6761-0c7c-405f-bc77-b40d6563133e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:26:30.168838 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcf6761-0c7c-405f-bc77-b40d6563133e service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:26:30.187665 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 719/1443] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:26:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:26:32.426589 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd59b7e-4b50-4235-8d0a-304723051244 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:32.427152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd59b7e-4b50-4235-8d0a-304723051244 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:32.427152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd59b7e-4b50-4235-8d0a-304723051244 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:32.428458 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 725/1444] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:26:32.433779 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d36ff-ef8c-43c7-b5be-a3f901eb3dde None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:32.434205 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d36ff-ef8c-43c7-b5be-a3f901eb3dde None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:32.434348 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d36ff-ef8c-43c7-b5be-a3f901eb3dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:32.500154 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-bf9d36ff-ef8c-43c7-b5be-a3f901eb3dde None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:26:32.501345 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bf9d36ff-ef8c-43c7-b5be-a3f901eb3dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:32.513935 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:32.595314 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 720/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:26:32.876413 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd59b7e-4b50-4235-8d0a-304723051244 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:26:32.877960 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9cd59b7e-4b50-4235-8d0a-304723051244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:32.979556 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd59b7e-4b50-4235-8d0a-304723051244 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:26:32.992130 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:33.052231 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ef425-bef7-4d66-8032-ab5ffdf1cb30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:26:33.052830 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ef425-bef7-4d66-8032-ab5ffdf1cb30 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:33.053105 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ef425-bef7-4d66-8032-ab5ffdf1cb30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:33.053375 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ef425-bef7-4d66-8032-ab5ffdf1cb30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:33.055236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-873ef425-bef7-4d66-8032-ab5ffdf1cb30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:33.162441 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ef425-bef7-4d66-8032-ab5ffdf1cb30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:26:33.163827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ef425-bef7-4d66-8032-ab5ffdf1cb30 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:26:33.183993 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 726/1446] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:26:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:26:35.547674 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff13d8-db5d-4942-888c-c217457c1a39 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:35.548242 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff13d8-db5d-4942-888c-c217457c1a39 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:35.548242 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff13d8-db5d-4942-888c-c217457c1a39 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:35.549591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 721/1447] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:26:35.554729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2b09d-d607-4f27-a10b-e2665397555e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:35.555154 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2b09d-d607-4f27-a10b-e2665397555e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:35.555297 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2b09d-d607-4f27-a10b-e2665397555e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:35.603325 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:35.621247 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-bff2b09d-d607-4f27-a10b-e2665397555e None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:26:35.622456 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bff2b09d-d607-4f27-a10b-e2665397555e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:35.718149 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 727/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:26:36.006192 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff13d8-db5d-4942-888c-c217457c1a39 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:26:36.007764 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b3ff13d8-db5d-4942-888c-c217457c1a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:36.109583 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff13d8-db5d-4942-888c-c217457c1a39 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:26:36.149204 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:36.182877 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92151cd-bd8a-460e-9a22-c948176cd4c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:26:36.183483 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92151cd-bd8a-460e-9a22-c948176cd4c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:36.183756 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92151cd-bd8a-460e-9a22-c948176cd4c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:36.184020 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92151cd-bd8a-460e-9a22-c948176cd4c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:36.185594 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d92151cd-bd8a-460e-9a22-c948176cd4c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:36.291330 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92151cd-bd8a-460e-9a22-c948176cd4c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:26:36.292770 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92151cd-bd8a-460e-9a22-c948176cd4c9 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:26:36.309950 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 722/1449] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:26:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:26:38.338415 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027eb38d-3e71-4d0b-a966-5229ee6d541b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:38.338956 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027eb38d-3e71-4d0b-a966-5229ee6d541b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:38.338956 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027eb38d-3e71-4d0b-a966-5229ee6d541b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:38.340280 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 728/1450] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:26:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:26:38.345646 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cf0c4-26f7-494d-ac85-a7fe4f481225 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:38.346031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cf0c4-26f7-494d-ac85-a7fe4f481225 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:38.346196 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612cf0c4-26f7-494d-ac85-a7fe4f481225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:38.411323 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-612cf0c4-26f7-494d-ac85-a7fe4f481225 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:26:38.412520 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-612cf0c4-26f7-494d-ac85-a7fe4f481225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:38.432433 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:38.507958 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 723/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:26:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:26:38.790337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027eb38d-3e71-4d0b-a966-5229ee6d541b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:26:38.791898 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-027eb38d-3e71-4d0b-a966-5229ee6d541b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:38.894241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027eb38d-3e71-4d0b-a966-5229ee6d541b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:26:38.915190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:38.969714 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f52bf87-1ab4-4128-b332-57732b40ddbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:26:38.970328 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52bf87-1ab4-4128-b332-57732b40ddbd service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:38.970608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52bf87-1ab4-4128-b332-57732b40ddbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:38.970913 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f52bf87-1ab4-4128-b332-57732b40ddbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:38.972516 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2f52bf87-1ab4-4128-b332-57732b40ddbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:39.079789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f52bf87-1ab4-4128-b332-57732b40ddbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:26:39.081142 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f52bf87-1ab4-4128-b332-57732b40ddbd service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:26:39.099701 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 729/1452] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:26:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:26:41.055165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5038f-4cf9-4941-a0dc-7bd13b0b08c9 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:41.055720 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5038f-4cf9-4941-a0dc-7bd13b0b08c9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:41.055720 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5038f-4cf9-4941-a0dc-7bd13b0b08c9 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:41.057184 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 724/1453] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:26:41.062516 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f35cd-3fe1-41a4-93d4-07e80ce03aaf None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:41.062968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f35cd-3fe1-41a4-93d4-07e80ce03aaf None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:41.063133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f35cd-3fe1-41a4-93d4-07e80ce03aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:41.126767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:41.129371 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b81f35cd-3fe1-41a4-93d4-07e80ce03aaf None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:26:41.130560 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b81f35cd-3fe1-41a4-93d4-07e80ce03aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:41.224732 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 730/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:26:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:26:41.512836 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5038f-4cf9-4941-a0dc-7bd13b0b08c9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:26:41.514897 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5cd5038f-4cf9-4941-a0dc-7bd13b0b08c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:41.618237 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5038f-4cf9-4941-a0dc-7bd13b0b08c9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:26:41.666954 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:41.690594 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43bfa8f-5365-4bf7-b208-67cc43446740 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:26:41.691222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bfa8f-5365-4bf7-b208-67cc43446740 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:41.691501 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bfa8f-5365-4bf7-b208-67cc43446740 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:41.691771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bfa8f-5365-4bf7-b208-67cc43446740 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:41.693390 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a43bfa8f-5365-4bf7-b208-67cc43446740 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:41.800938 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43bfa8f-5365-4bf7-b208-67cc43446740 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:26:41.802289 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43bfa8f-5365-4bf7-b208-67cc43446740 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:26:41.819685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 725/1455] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:26:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:26:43.758453 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f8d6b-70ef-4a57-9485-098a862fd1f5 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:43.759091 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f8d6b-70ef-4a57-9485-098a862fd1f5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:43.759091 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f8d6b-70ef-4a57-9485-098a862fd1f5 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:43.760404 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 731/1456] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:26:43.766061 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273dd1d3-f0fc-45f9-8f82-5a388dee485b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:43.766520 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273dd1d3-f0fc-45f9-8f82-5a388dee485b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:43.766635 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273dd1d3-f0fc-45f9-8f82-5a388dee485b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:43.831951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-273dd1d3-f0fc-45f9-8f82-5a388dee485b None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:26:43.833165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-273dd1d3-f0fc-45f9-8f82-5a388dee485b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:43.861986 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:43.929020 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 726/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:26:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:26:44.214465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4f8d6b-70ef-4a57-9485-098a862fd1f5 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:26:44.216093 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ff4f8d6b-70ef-4a57-9485-098a862fd1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:44.318851 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4f8d6b-70ef-4a57-9485-098a862fd1f5 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:26:44.346920 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:44.392364 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5159e92-8078-484b-a779-b83631e28b50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:26:44.392981 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5159e92-8078-484b-a779-b83631e28b50 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:44.393598 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5159e92-8078-484b-a779-b83631e28b50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:44.393869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5159e92-8078-484b-a779-b83631e28b50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:44.395498 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d5159e92-8078-484b-a779-b83631e28b50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:44.506959 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5159e92-8078-484b-a779-b83631e28b50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:26:44.508389 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5159e92-8078-484b-a779-b83631e28b50 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:26:44.526480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 732/1458] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:26:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:26:46.471466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d0662-fbc5-4501-8240-297952010d13 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:46.472026 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d0662-fbc5-4501-8240-297952010d13 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:46.472026 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d0662-fbc5-4501-8240-297952010d13 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:46.473346 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 727/1459] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:26:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:26:46.478716 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f6eec-a87d-41f0-bf36-43943e5edf76 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:46.479123 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f6eec-a87d-41f0-bf36-43943e5edf76 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:46.479281 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f6eec-a87d-41f0-bf36-43943e5edf76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:46.547468 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d13f6eec-a87d-41f0-bf36-43943e5edf76 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:26:46.548681 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d13f6eec-a87d-41f0-bf36-43943e5edf76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:46.556987 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:46.765110 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 733/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:26:47.054169 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d0662-fbc5-4501-8240-297952010d13 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:26:47.055768 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b13d0662-fbc5-4501-8240-297952010d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:47.157373 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d0662-fbc5-4501-8240-297952010d13 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:26:47.211407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:47.229633 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1d775-4bf9-4971-8c8e-d86db6663694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:26:47.230236 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1d775-4bf9-4971-8c8e-d86db6663694 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:47.230514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1d775-4bf9-4971-8c8e-d86db6663694 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:47.230815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1d775-4bf9-4971-8c8e-d86db6663694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:47.232401 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b9f1d775-4bf9-4971-8c8e-d86db6663694 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:47.338362 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1d775-4bf9-4971-8c8e-d86db6663694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:26:47.339737 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1d775-4bf9-4971-8c8e-d86db6663694 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:26:47.359847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 728/1461] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:26:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:26:49.304166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82417ea-d788-4daa-a4bd-2aa805cd0d85 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:49.304676 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82417ea-d788-4daa-a4bd-2aa805cd0d85 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:49.304676 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82417ea-d788-4daa-a4bd-2aa805cd0d85 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:49.305976 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 734/1462] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:26:49.311273 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac75cb4-638f-4e47-a52f-4ac33426a79d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:49.311707 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac75cb4-638f-4e47-a52f-4ac33426a79d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:49.311816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac75cb4-638f-4e47-a52f-4ac33426a79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:49.378685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5ac75cb4-638f-4e47-a52f-4ac33426a79d None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:26:49.379898 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5ac75cb4-638f-4e47-a52f-4ac33426a79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:49.417052 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:49.474577 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 729/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:26:49.763358 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82417ea-d788-4daa-a4bd-2aa805cd0d85 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:26:49.764919 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d82417ea-d788-4daa-a4bd-2aa805cd0d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:49.868638 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82417ea-d788-4daa-a4bd-2aa805cd0d85 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:26:49.905967 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:49.941362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9cd8cb-0f54-4213-ac80-f9c091303d13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:26:49.941963 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9cd8cb-0f54-4213-ac80-f9c091303d13 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:49.942293 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9cd8cb-0f54-4213-ac80-f9c091303d13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:49.942564 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9cd8cb-0f54-4213-ac80-f9c091303d13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:49.944198 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cf9cd8cb-0f54-4213-ac80-f9c091303d13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:50.051373 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9cd8cb-0f54-4213-ac80-f9c091303d13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:26:50.052714 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9cd8cb-0f54-4213-ac80-f9c091303d13 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:26:50.070783 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 735/1464] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:26:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:26:52.016639 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4f576-97dc-4dee-81b9-5c4e09a010e2 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:52.017191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4f576-97dc-4dee-81b9-5c4e09a010e2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:52.017191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4f576-97dc-4dee-81b9-5c4e09a010e2 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:52.018559 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 730/1465] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:26:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:26:52.023767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb0ad0-015d-40c7-8a4b-65882d1204b6 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:52.024234 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb0ad0-015d-40c7-8a4b-65882d1204b6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:52.024350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb0ad0-015d-40c7-8a4b-65882d1204b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:52.089338 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f4cb0ad0-015d-40c7-8a4b-65882d1204b6 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:26:52.090544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f4cb0ad0-015d-40c7-8a4b-65882d1204b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:52.109288 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:52.195060 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 736/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:26:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:26:52.481765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4f576-97dc-4dee-81b9-5c4e09a010e2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:26:52.483343 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c2c4f576-97dc-4dee-81b9-5c4e09a010e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:52.584241 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4f576-97dc-4dee-81b9-5c4e09a010e2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:26:52.649425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:52.656858 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d291d0-c2ed-4206-ac8f-e5d7a8fbb920 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:26:52.661120 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d291d0-c2ed-4206-ac8f-e5d7a8fbb920 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:52.661120 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d291d0-c2ed-4206-ac8f-e5d7a8fbb920 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:52.661120 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d291d0-c2ed-4206-ac8f-e5d7a8fbb920 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:52.661120 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d4d291d0-c2ed-4206-ac8f-e5d7a8fbb920 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:52.766727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d291d0-c2ed-4206-ac8f-e5d7a8fbb920 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:26:52.768103 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d291d0-c2ed-4206-ac8f-e5d7a8fbb920 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:26:52.786654 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 731/1467] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:26:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:26:54.697116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ebb69-2d1d-4828-a62f-6261646bc6ae None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:54.697675 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ebb69-2d1d-4828-a62f-6261646bc6ae None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:54.697675 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ebb69-2d1d-4828-a62f-6261646bc6ae None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:54.699053 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 737/1468] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:26:54.704872 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5b23a-1170-4859-b15a-1cb3d75fdbf0 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:54.705311 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5b23a-1170-4859-b15a-1cb3d75fdbf0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:54.705419 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb5b23a-1170-4859-b15a-1cb3d75fdbf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:54.774903 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2bb5b23a-1170-4859-b15a-1cb3d75fdbf0 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:26:54.776076 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2bb5b23a-1170-4859-b15a-1cb3d75fdbf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:54.829254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:54.882821 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 732/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:26:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:26:55.165684 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7ebb69-2d1d-4828-a62f-6261646bc6ae None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:26:55.167228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fc7ebb69-2d1d-4828-a62f-6261646bc6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:55.267392 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7ebb69-2d1d-4828-a62f-6261646bc6ae None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:26:55.313430 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:26:55.338935 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17608444-9788-47ed-b9fb-ee2ace202eb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:26:55.339530 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17608444-9788-47ed-b9fb-ee2ace202eb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:26:55.339802 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17608444-9788-47ed-b9fb-ee2ace202eb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:26:55.340071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17608444-9788-47ed-b9fb-ee2ace202eb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:26:55.342361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-17608444-9788-47ed-b9fb-ee2ace202eb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:26:55.446560 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17608444-9788-47ed-b9fb-ee2ace202eb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:26:55.447949 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17608444-9788-47ed-b9fb-ee2ace202eb7 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:26:55.468389 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 738/1470] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:26:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:27:04.880960 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=22 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:27:05.466081 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=66 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:27:14.830454 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81166815-da10-4c76-b0c1-c0b3ecdcd5ba None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:27:14.831031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81166815-da10-4c76-b0c1-c0b3ecdcd5ba None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:27:14.831031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81166815-da10-4c76-b0c1-c0b3ecdcd5ba None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:27:14.832358 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 733/1471] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:27:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:27:14.837547 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d79876-7a51-4f72-82bb-eedc25263afc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:27:14.837986 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d79876-7a51-4f72-82bb-eedc25263afc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:27:14.838090 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d79876-7a51-4f72-82bb-eedc25263afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:27:14.903743 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b4d79876-7a51-4f72-82bb-eedc25263afc None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:27:14.905024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b4d79876-7a51-4f72-82bb-eedc25263afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:27:14.998833 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 739/1472] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:27:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:27:15.290437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98711bc-aeef-4495-8ba9-cc0e8b710a67 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:27:15.290743 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98711bc-aeef-4495-8ba9-cc0e8b710a67 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:27:15.291024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98711bc-aeef-4495-8ba9-cc0e8b710a67 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:27:15.292363 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 734/1473] 10.222.0.57 () {54 vars in 961 bytes} [Fri Sep 2 09:27:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:27:15.297648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482057b7-35a1-4031-93da-d661ef4a7d5c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:27:15.297942 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482057b7-35a1-4031-93da-d661ef4a7d5c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:27:15.298219 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482057b7-35a1-4031-93da-d661ef4a7d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:27:15.375436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-482057b7-35a1-4031-93da-d661ef4a7d5c None None] MFA Rules not processed for user `31b555bb94ce44ce918cbf2a012576a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:27:15.376627 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-482057b7-35a1-4031-93da-d661ef4a7d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:27:15.396974 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:27:15.449999 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 740/1474] 10.222.0.57 () {58 vars in 1058 bytes} [Fri Sep 2 09:27:15 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:27:15.458056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98711bc-aeef-4495-8ba9-cc0e8b710a67 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:27:15.459632 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a98711bc-aeef-4495-8ba9-cc0e8b710a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:27:15.563101 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98711bc-aeef-4495-8ba9-cc0e8b710a67 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:27:15.636158 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234004ae-e55a-4e44-90ab-53f61afd579a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b555bb94ce44ce918cbf2a012576a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:27:15.636761 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234004ae-e55a-4e44-90ab-53f61afd579a service glance] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:27:15.637034 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234004ae-e55a-4e44-90ab-53f61afd579a service glance] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:27:15.637299 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234004ae-e55a-4e44-90ab-53f61afd579a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:27:15.639530 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-234004ae-e55a-4e44-90ab-53f61afd579a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:27:15.692253 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:27:15.746791 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234004ae-e55a-4e44-90ab-53f61afd579a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:27:15.748220 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234004ae-e55a-4e44-90ab-53f61afd579a service glance] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:27:15.767730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 735/1475] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:27:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:27:17.731578 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130ce53-607d-40e1-b959-fb525bec654b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:27:17.732150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130ce53-607d-40e1-b959-fb525bec654b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:27:17.732150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130ce53-607d-40e1-b959-fb525bec654b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:27:17.733515 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 741/1476] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:27:17.738506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859307a-5930-40ad-912c-ffef97d2987a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:27:17.738959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859307a-5930-40ad-912c-ffef97d2987a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:27:17.739070 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859307a-5930-40ad-912c-ffef97d2987a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:27:17.804387 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6859307a-5930-40ad-912c-ffef97d2987a None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:27:17.805596 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6859307a-5930-40ad-912c-ffef97d2987a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:27:17.904394 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 736/1477] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:27:18.173649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130ce53-607d-40e1-b959-fb525bec654b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:27:18.175209 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7130ce53-607d-40e1-b959-fb525bec654b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:27:18.296265 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130ce53-607d-40e1-b959-fb525bec654b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:27:18.343579 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:27:18.387438 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007025fc-8ce9-4d0c-963f-6830f6da2c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:27:18.388043 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007025fc-8ce9-4d0c-963f-6830f6da2c1e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:27:18.388810 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007025fc-8ce9-4d0c-963f-6830f6da2c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:27:18.389079 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007025fc-8ce9-4d0c-963f-6830f6da2c1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:27:18.389710 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007025fc-8ce9-4d0c-963f-6830f6da2c1e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:27:18.391474 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-007025fc-8ce9-4d0c-963f-6830f6da2c1e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:27:18.391474 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Sep 02 09:27:18.391474 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: Sep 02 09:27:18.392599 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007025fc-8ce9-4d0c-963f-6830f6da2c1e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:27:18.411782 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 742/1478] 10.222.0.57 () {56 vars in 1108 bytes} [Fri Sep 2 09:27:18 2022] GET /identity/v3/projects => generated 2553 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:27:20.313191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf944856-9ba0-46cf-b4f9-df9640ee4da9 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:27:20.313763 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf944856-9ba0-46cf-b4f9-df9640ee4da9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:27:20.313763 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf944856-9ba0-46cf-b4f9-df9640ee4da9 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:27:20.315129 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 737/1479] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:27:20.320176 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426b731-0af8-4143-89b4-6ca0404030a5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:27:20.320603 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426b731-0af8-4143-89b4-6ca0404030a5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:27:20.320750 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426b731-0af8-4143-89b4-6ca0404030a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:27:20.384844 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5426b731-0af8-4143-89b4-6ca0404030a5 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:27:20.386045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5426b731-0af8-4143-89b4-6ca0404030a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:27:20.483104 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 743/1480] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:27:20.777738 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf944856-9ba0-46cf-b4f9-df9640ee4da9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:27:20.779310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bf944856-9ba0-46cf-b4f9-df9640ee4da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:27:20.789028 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:27:20.926364 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf944856-9ba0-46cf-b4f9-df9640ee4da9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:27:21.007969 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:27:21.041579 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02ceb75-0d4a-49de-a846-803643403a13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:27:21.042114 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ceb75-0d4a-49de-a846-803643403a13 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:27:21.042391 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ceb75-0d4a-49de-a846-803643403a13 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:27:21.042661 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ceb75-0d4a-49de-a846-803643403a13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:27:21.045008 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e02ceb75-0d4a-49de-a846-803643403a13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:27:21.131900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02ceb75-0d4a-49de-a846-803643403a13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:27:21.133536 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02ceb75-0d4a-49de-a846-803643403a13 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:27:21.151306 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 738/1481] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:27:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:27:23.267017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac4761-83d3-4de2-a3e5-8d9b6d6839a7 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:27:23.267623 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac4761-83d3-4de2-a3e5-8d9b6d6839a7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:27:23.267623 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac4761-83d3-4de2-a3e5-8d9b6d6839a7 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:27:23.268906 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 744/1482] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:27:23.274157 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468e7a2-5493-4b0b-8a4e-bae45635c451 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:27:23.274607 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468e7a2-5493-4b0b-8a4e-bae45635c451 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:27:23.274755 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468e7a2-5493-4b0b-8a4e-bae45635c451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:27:23.340220 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d468e7a2-5493-4b0b-8a4e-bae45635c451 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:27:23.341451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d468e7a2-5493-4b0b-8a4e-bae45635c451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:27:23.386617 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:27:23.438359 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 739/1483] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:27:23.725135 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ac4761-83d3-4de2-a3e5-8d9b6d6839a7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:27:23.726664 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-54ac4761-83d3-4de2-a3e5-8d9b6d6839a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:27:23.749548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:27:23.829071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ac4761-83d3-4de2-a3e5-8d9b6d6839a7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:27:23.902247 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd67ef-a4b3-4db3-9706-67fcb292255a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:27:23.902900 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd67ef-a4b3-4db3-9706-67fcb292255a service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:27:23.903188 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd67ef-a4b3-4db3-9706-67fcb292255a service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:27:23.903629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd67ef-a4b3-4db3-9706-67fcb292255a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:27:23.905261 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a4bd67ef-a4b3-4db3-9706-67fcb292255a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:27:23.979620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:27:24.013947 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd67ef-a4b3-4db3-9706-67fcb292255a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:27:24.015429 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd67ef-a4b3-4db3-9706-67fcb292255a service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:27:24.033771 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 745/1484] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:27:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:27:25.980495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f9473-cd02-4574-9c34-1c479891fe8c None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:27:25.981055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f9473-cd02-4574-9c34-1c479891fe8c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:27:25.981055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760f9473-cd02-4574-9c34-1c479891fe8c None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:27:25.982450 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 740/1485] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:27:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:27:25.987547 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c094f06-6c6f-4f80-814b-2f47feaed596 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:27:25.987956 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c094f06-6c6f-4f80-814b-2f47feaed596 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:27:25.988102 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c094f06-6c6f-4f80-814b-2f47feaed596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:27:26.052948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7c094f06-6c6f-4f80-814b-2f47feaed596 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:27:26.054080 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7c094f06-6c6f-4f80-814b-2f47feaed596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:27:26.149125 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 746/1486] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:27:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:27:26.430917 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f9473-cd02-4574-9c34-1c479891fe8c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:27:26.432432 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-760f9473-cd02-4574-9c34-1c479891fe8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:27:26.532820 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760f9473-cd02-4574-9c34-1c479891fe8c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:27:26.603394 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:27:26.605244 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40bd46-c255-4b39-9ff0-ef7fa95514ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:27:26.605846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40bd46-c255-4b39-9ff0-ef7fa95514ed service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:27:26.606121 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40bd46-c255-4b39-9ff0-ef7fa95514ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:27:26.606919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40bd46-c255-4b39-9ff0-ef7fa95514ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:27:26.608493 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4f40bd46-c255-4b39-9ff0-ef7fa95514ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:27:26.716203 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f40bd46-c255-4b39-9ff0-ef7fa95514ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:27:26.717633 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f40bd46-c255-4b39-9ff0-ef7fa95514ed service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:27:26.735910 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 741/1487] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:27:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:27:28.868464 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a0c77-1ec2-4831-bdf4-8050861f7331 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:27:28.868906 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a0c77-1ec2-4831-bdf4-8050861f7331 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:27:28.869018 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a0c77-1ec2-4831-bdf4-8050861f7331 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:27:28.870386 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 747/1488] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:27:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:27:28.876361 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1675415-65ec-44c3-b029-534b2c1bcde3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:27:28.876841 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1675415-65ec-44c3-b029-534b2c1bcde3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:27:28.876951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1675415-65ec-44c3-b029-534b2c1bcde3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:27:28.941052 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e1675415-65ec-44c3-b029-534b2c1bcde3 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:27:28.942246 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e1675415-65ec-44c3-b029-534b2c1bcde3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:27:28.996361 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:27:29.039888 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 742/1489] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:27:29.329968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a0c77-1ec2-4831-bdf4-8050861f7331 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:27:29.331614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-419a0c77-1ec2-4831-bdf4-8050861f7331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:27:29.367583 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:27:29.435153 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a0c77-1ec2-4831-bdf4-8050861f7331 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:27:29.505694 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5259c4d2-ca0a-4664-b3e1-d73be6d24e24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:27:29.506293 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259c4d2-ca0a-4664-b3e1-d73be6d24e24 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:27:29.506566 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259c4d2-ca0a-4664-b3e1-d73be6d24e24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:27:29.506863 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5259c4d2-ca0a-4664-b3e1-d73be6d24e24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:27:29.508429 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5259c4d2-ca0a-4664-b3e1-d73be6d24e24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:27:29.587104 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:27:29.616810 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5259c4d2-ca0a-4664-b3e1-d73be6d24e24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:27:29.618232 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5259c4d2-ca0a-4664-b3e1-d73be6d24e24 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:27:29.636084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 748/1490] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:27:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:27:31.914664 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2572cb3-a001-41af-b7c5-df5b6da9bd2c None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:27:31.915216 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2572cb3-a001-41af-b7c5-df5b6da9bd2c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:27:31.915335 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2572cb3-a001-41af-b7c5-df5b6da9bd2c None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:27:31.916626 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 743/1491] 10.222.0.57 () {54 vars in 927 bytes} [Fri Sep 2 09:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:27:31.922332 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d66547-dfd0-4001-8aac-8a1455048414 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:27:31.922760 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d66547-dfd0-4001-8aac-8a1455048414 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:27:31.922915 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d66547-dfd0-4001-8aac-8a1455048414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:27:31.991061 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c0d66547-dfd0-4001-8aac-8a1455048414 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:27:31.992282 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c0d66547-dfd0-4001-8aac-8a1455048414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:27:32.087976 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 749/1492] 10.222.0.57 () {58 vars in 1024 bytes} [Fri Sep 2 09:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:27:32.376799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2572cb3-a001-41af-b7c5-df5b6da9bd2c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:27:32.378317 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e2572cb3-a001-41af-b7c5-df5b6da9bd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:27:32.480602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2572cb3-a001-41af-b7c5-df5b6da9bd2c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:27:32.552764 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d918d4-679f-4478-80aa-efb82479c512 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:27:32.553404 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d918d4-679f-4478-80aa-efb82479c512 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:27:32.553682 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d918d4-679f-4478-80aa-efb82479c512 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:27:32.553948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d918d4-679f-4478-80aa-efb82479c512 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:27:32.555574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c1d918d4-679f-4478-80aa-efb82479c512 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:27:32.564721 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:27:32.663273 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d918d4-679f-4478-80aa-efb82479c512 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:27:32.664638 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d918d4-679f-4478-80aa-efb82479c512 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:27:32.682901 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 744/1493] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:27:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:27:42.086173 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=82 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:27:42.680604 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=50 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:28:53.413141 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f35dcf-04f1-40f9-8e05-3c8f1d2a390b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:28:53.413716 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f35dcf-04f1-40f9-8e05-3c8f1d2a390b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:28:53.413716 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f35dcf-04f1-40f9-8e05-3c8f1d2a390b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:28:53.489092 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-67f35dcf-04f1-40f9-8e05-3c8f1d2a390b None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:28:53.490298 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-67f35dcf-04f1-40f9-8e05-3c8f1d2a390b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:28:53.584590 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 750/1494] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:28:53.592688 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c6ead-711d-4091-80ee-788120439572 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:28:53.593179 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c6ead-711d-4091-80ee-788120439572 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:28:53.593275 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c6ead-711d-4091-80ee-788120439572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:28:53.668027 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b68c6ead-711d-4091-80ee-788120439572 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:28:53.669170 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b68c6ead-711d-4091-80ee-788120439572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:28:53.763502 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 745/1495] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:28:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:28:53.768017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f35dcf-04f1-40f9-8e05-3c8f1d2a390b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:28:53.769206 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-67f35dcf-04f1-40f9-8e05-3c8f1d2a390b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:28:53.781328 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f35dcf-04f1-40f9-8e05-3c8f1d2a390b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:28:53.840522 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:28:53.873324 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0418637f-59a3-468a-a2b6-e662e7d032f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:28:53.874044 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418637f-59a3-468a-a2b6-e662e7d032f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:28:53.874266 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418637f-59a3-468a-a2b6-e662e7d032f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:28:53.874532 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418637f-59a3-468a-a2b6-e662e7d032f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:28:53.875201 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0418637f-59a3-468a-a2b6-e662e7d032f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:28:53.875501 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0418637f-59a3-468a-a2b6-e662e7d032f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:28:53.876506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-0418637f-59a3-468a-a2b6-e662e7d032f2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:28:53.876506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Sep 02 09:28:53.876506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: Sep 02 09:28:53.877412 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0418637f-59a3-468a-a2b6-e662e7d032f2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:28:53.889400 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 751/1496] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:28:53.897978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c6ead-711d-4091-80ee-788120439572 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:28:53.907054 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c6ead-711d-4091-80ee-788120439572 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:28:53.967168 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:28:54.000777 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde6517-7307-4482-b0bc-7750ba30bbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:28:54.001385 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde6517-7307-4482-b0bc-7750ba30bbb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:28:54.001657 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde6517-7307-4482-b0bc-7750ba30bbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:28:54.001925 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cde6517-7307-4482-b0bc-7750ba30bbb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:28:54.002553 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde6517-7307-4482-b0bc-7750ba30bbb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:28:54.004329 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: WARNING py.warnings [None req-0cde6517-7307-4482-b0bc-7750ba30bbb7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:28:54.004329 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: warnings.warn(msg) Sep 02 09:28:54.004329 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: Sep 02 09:28:54.005505 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cde6517-7307-4482-b0bc-7750ba30bbb7 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:28:54.239782 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 746/1497] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:28:53 2022] POST /identity/v3/projects => generated 379 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:28:54.243933 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0418637f-59a3-468a-a2b6-e662e7d032f2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:28:54.246163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0418637f-59a3-468a-a2b6-e662e7d032f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:28:54.279250 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0418637f-59a3-468a-a2b6-e662e7d032f2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:28:54.372866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1b654-bb4d-40d3-b0c6-4c9895ce95ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:28:54.374330 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1b654-bb4d-40d3-b0c6-4c9895ce95ef admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:28:54.374618 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1b654-bb4d-40d3-b0c6-4c9895ce95ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:28:54.374927 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d1b654-bb4d-40d3-b0c6-4c9895ce95ef admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:28:54.375574 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d1b654-bb4d-40d3-b0c6-4c9895ce95ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:28:54.376642 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-59d1b654-bb4d-40d3-b0c6-4c9895ce95ef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:28:54.376642 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Sep 02 09:28:54.376642 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: Sep 02 09:28:54.377848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d1b654-bb4d-40d3-b0c6-4c9895ce95ef admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:28:54.485822 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 752/1498] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:28:54 2022] POST /identity/v3/users => generated 290 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:28:54.490004 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde6517-7307-4482-b0bc-7750ba30bbb7 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:28:54.491576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0cde6517-7307-4482-b0bc-7750ba30bbb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:28:54.553852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cde6517-7307-4482-b0bc-7750ba30bbb7 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:28:54.636291 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:28:54.646903 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90debe-5c7e-45d0-8797-7cb1bdb63812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:28:54.651204 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90debe-5c7e-45d0-8797-7cb1bdb63812 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:28:54.651204 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90debe-5c7e-45d0-8797-7cb1bdb63812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:28:54.651204 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90debe-5c7e-45d0-8797-7cb1bdb63812 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:28:54.651204 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f90debe-5c7e-45d0-8797-7cb1bdb63812 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:28:54.651204 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: WARNING py.warnings [None req-7f90debe-5c7e-45d0-8797-7cb1bdb63812 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:28:54.651204 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: warnings.warn(msg) Sep 02 09:28:54.651204 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: Sep 02 09:28:54.651766 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f90debe-5c7e-45d0-8797-7cb1bdb63812 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:28:54.663084 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 747/1499] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:28:54.667867 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1b654-bb4d-40d3-b0c6-4c9895ce95ef admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:28:54.669366 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-59d1b654-bb4d-40d3-b0c6-4c9895ce95ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:28:54.680578 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d1b654-bb4d-40d3-b0c6-4c9895ce95ef admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:28:54.704875 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:28:54.774337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a19da43-77f8-412f-a013-b8b2edcb9f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:28:54.774955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19da43-77f8-412f-a013-b8b2edcb9f06 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:28:54.775233 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19da43-77f8-412f-a013-b8b2edcb9f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:28:54.775991 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a19da43-77f8-412f-a013-b8b2edcb9f06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:28:54.776633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a19da43-77f8-412f-a013-b8b2edcb9f06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:28:54.777645 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-1a19da43-77f8-412f-a013-b8b2edcb9f06 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:28:54.777645 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Sep 02 09:28:54.777645 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: Sep 02 09:28:54.778533 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a19da43-77f8-412f-a013-b8b2edcb9f06 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:28:54.791001 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 753/1500] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:28:54 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:28:54.795741 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90debe-5c7e-45d0-8797-7cb1bdb63812 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:28:54.797247 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7f90debe-5c7e-45d0-8797-7cb1bdb63812 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:28:54.830298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90debe-5c7e-45d0-8797-7cb1bdb63812 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:28:54.926563 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48425544-207f-420d-b8c9-9214a529e92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:28:54.927251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48425544-207f-420d-b8c9-9214a529e92e admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:28:54.927529 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48425544-207f-420d-b8c9-9214a529e92e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:28:54.927802 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48425544-207f-420d-b8c9-9214a529e92e admin admin] PATH_INFO: `/v3/projects/1d0f8429e2ba4fe887f510c2b7e433c4/users/1b56ca822b4b488d8af50861af632ece/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:28:54.949880 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48425544-207f-420d-b8c9-9214a529e92e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d0f8429e2ba4fe887f510c2b7e433c4, user_id=1b56ca822b4b488d8af50861af632ece, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:28:54.950935 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: WARNING py.warnings [None req-48425544-207f-420d-b8c9-9214a529e92e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:28:54.950935 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: warnings.warn(msg) Sep 02 09:28:54.950935 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: Sep 02 09:28:54.952876 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48425544-207f-420d-b8c9-9214a529e92e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:28:55.044081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 748/1501] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:28:54 2022] PUT /identity/v3/projects/1d0f8429e2ba4fe887f510c2b7e433c4/users/1b56ca822b4b488d8af50861af632ece/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:28:55.056237 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6154f-2f83-413a-942a-1eb255ae8de6 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:28:55.056517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6154f-2f83-413a-942a-1eb255ae8de6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:28:55.057443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a6154f-2f83-413a-942a-1eb255ae8de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:28:55.112914 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-95a6154f-2f83-413a-942a-1eb255ae8de6 None None] MFA Rules not processed for user `1b56ca822b4b488d8af50861af632ece`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:28:55.114108 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-95a6154f-2f83-413a-942a-1eb255ae8de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:28:55.189407 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 754/1502] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:28:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:28:55.202531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb9fb0-b911-439a-a88f-71ec3bc7b908 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:28:55.202854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb9fb0-b911-439a-a88f-71ec3bc7b908 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:28:55.203131 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb9fb0-b911-439a-a88f-71ec3bc7b908 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:28:55.204476 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 749/1503] 10.222.0.57 () {54 vars in 964 bytes} [Fri Sep 2 09:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:28:55.209164 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9666a3a-bba0-4dad-af07-e824f4ad347e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:28:55.209495 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9666a3a-bba0-4dad-af07-e824f4ad347e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:28:55.209769 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9666a3a-bba0-4dad-af07-e824f4ad347e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:28:55.241271 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:28:55.283915 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f9666a3a-bba0-4dad-af07-e824f4ad347e None None] MFA Rules not processed for user `b6a1f44f6fe94c09b696867871fd5b47`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:28:55.285105 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f9666a3a-bba0-4dad-af07-e824f4ad347e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:28:55.360424 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 755/1504] 10.222.0.57 () {58 vars in 1061 bytes} [Fri Sep 2 09:28:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:28:55.370212 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb9fb0-b911-439a-a88f-71ec3bc7b908 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:28:55.371801 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-33eb9fb0-b911-439a-a88f-71ec3bc7b908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:28:55.424144 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:28:55.480564 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb9fb0-b911-439a-a88f-71ec3bc7b908 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:28:55.552367 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad81f0e-f883-474a-bfea-790a0716d88d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:28:55.552968 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad81f0e-f883-474a-bfea-790a0716d88d service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:28:55.553244 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad81f0e-f883-474a-bfea-790a0716d88d service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:28:55.553511 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad81f0e-f883-474a-bfea-790a0716d88d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:28:55.555152 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-dad81f0e-f883-474a-bfea-790a0716d88d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:28:55.652321 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad81f0e-f883-474a-bfea-790a0716d88d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:28:55.653712 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad81f0e-f883-474a-bfea-790a0716d88d service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:28:55.657241 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:28:55.671035 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 750/1505] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:28:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:28:55.717164 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2af8f7-c1c1-4257-9b39-2cd343f7d0f9 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:28:55.717599 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2af8f7-c1c1-4257-9b39-2cd343f7d0f9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:28:55.717835 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2af8f7-c1c1-4257-9b39-2cd343f7d0f9 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:28:55.719144 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 756/1506] 10.222.0.57 () {52 vars in 830 bytes} [Fri Sep 2 09:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:28:55.722651 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad81f0e-f883-474a-bfea-790a0716d88d service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:28:55.723881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-dad81f0e-f883-474a-bfea-790a0716d88d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:28:55.819619 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad81f0e-f883-474a-bfea-790a0716d88d service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:28:55.918520 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4789c8-5383-4032-ae04-45d538cef999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:28:55.919115 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4789c8-5383-4032-ae04-45d538cef999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:28:55.919394 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4789c8-5383-4032-ae04-45d538cef999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:28:55.919680 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4789c8-5383-4032-ae04-45d538cef999 admin admin] PATH_INFO: `/v3/users/1b56ca822b4b488d8af50861af632ece` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:28:55.931671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4789c8-5383-4032-ae04-45d538cef999 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b56ca822b4b488d8af50861af632ece)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:28:55.932724 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: WARNING py.warnings [None req-1b4789c8-5383-4032-ae04-45d538cef999 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:28:55.932724 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: warnings.warn(msg) Sep 02 09:28:55.932724 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: Sep 02 09:28:55.933934 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4789c8-5383-4032-ae04-45d538cef999 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:28:55.943558 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:28:56.208586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1b4789c8-5383-4032-ae04-45d538cef999 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b56ca822b4b488d8af50861af632ece'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:28:56.223337 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1b4789c8-5383-4032-ae04-45d538cef999 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b56ca822b4b488d8af50861af632ece'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:28:56.279366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1b4789c8-5383-4032-ae04-45d538cef999 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b56ca822b4b488d8af50861af632ece'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:28:56.279798 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1b4789c8-5383-4032-ae04-45d538cef999 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b56ca822b4b488d8af50861af632ece'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:28:56.302248 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1b4789c8-5383-4032-ae04-45d538cef999 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b56ca822b4b488d8af50861af632ece'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:28:56.308350 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 751/1507] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:28:55 2022] DELETE /identity/v3/users/1b56ca822b4b488d8af50861af632ece => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:28:56.313023 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bc0f4-41d5-408a-b57e-c0c5b57c1e77 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:28:56.313315 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bc0f4-41d5-408a-b57e-c0c5b57c1e77 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:28:56.313592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2bc0f4-41d5-408a-b57e-c0c5b57c1e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:28:56.373283 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4b2bc0f4-41d5-408a-b57e-c0c5b57c1e77 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:28:56.374495 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4b2bc0f4-41d5-408a-b57e-c0c5b57c1e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:28:56.449209 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:28:56.461171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 757/1508] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:28:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:28:56.470611 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4789c8-5383-4032-ae04-45d538cef999 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:28:56.471881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1b4789c8-5383-4032-ae04-45d538cef999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:28:56.569902 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4789c8-5383-4032-ae04-45d538cef999 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:28:56.625059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:28:56.644448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90bf37-5783-4954-9446-895edd75f843 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:28:56.648868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90bf37-5783-4954-9446-895edd75f843 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:28:56.648868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90bf37-5783-4954-9446-895edd75f843 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:28:56.648868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90bf37-5783-4954-9446-895edd75f843 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:28:56.648868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bb90bf37-5783-4954-9446-895edd75f843 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:28:56.752970 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90bf37-5783-4954-9446-895edd75f843 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:28:56.754694 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90bf37-5783-4954-9446-895edd75f843 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:28:56.773503 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 752/1509] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:28:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:28:57.304412 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2bc0f4-41d5-408a-b57e-c0c5b57c1e77 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:28:57.305649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4b2bc0f4-41d5-408a-b57e-c0c5b57c1e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:28:57.317547 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2bc0f4-41d5-408a-b57e-c0c5b57c1e77 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:28:57.414940 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba65a3-1576-4cd4-8dd8-226e43d7f85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:28:57.415600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba65a3-1576-4cd4-8dd8-226e43d7f85a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:28:57.415882 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba65a3-1576-4cd4-8dd8-226e43d7f85a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:28:57.416154 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba65a3-1576-4cd4-8dd8-226e43d7f85a admin admin] PATH_INFO: `/v3/projects/1d0f8429e2ba4fe887f510c2b7e433c4` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:28:57.426090 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba65a3-1576-4cd4-8dd8-226e43d7f85a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d0f8429e2ba4fe887f510c2b7e433c4)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:28:57.427143 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-14ba65a3-1576-4cd4-8dd8-226e43d7f85a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:28:57.427143 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Sep 02 09:28:57.427143 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: Sep 02 09:28:57.428342 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba65a3-1576-4cd4-8dd8-226e43d7f85a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:28:57.545984 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-14ba65a3-1576-4cd4-8dd8-226e43d7f85a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d0f8429e2ba4fe887f510c2b7e433c4', 'request_id': 'req-14ba65a3-1576-4cd4-8dd8-226e43d7f85a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:28:57.552156 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-14ba65a3-1576-4cd4-8dd8-226e43d7f85a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d0f8429e2ba4fe887f510c2b7e433c4', 'request_id': 'req-14ba65a3-1576-4cd4-8dd8-226e43d7f85a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:28:57.565533 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-14ba65a3-1576-4cd4-8dd8-226e43d7f85a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d0f8429e2ba4fe887f510c2b7e433c4', 'request_id': 'req-14ba65a3-1576-4cd4-8dd8-226e43d7f85a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:28:57.667802 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-14ba65a3-1576-4cd4-8dd8-226e43d7f85a admin admin] The token cache is being invalidate because project 1d0f8429e2ba4fe887f510c2b7e433c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:28:57.668087 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-14ba65a3-1576-4cd4-8dd8-226e43d7f85a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:28:57.668526 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-14ba65a3-1576-4cd4-8dd8-226e43d7f85a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:28:57.669923 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 758/1510] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:28:57 2022] DELETE /identity/v3/projects/1d0f8429e2ba4fe887f510c2b7e433c4 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:29:06.771278 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=62 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:29:07.665712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:30:29.826110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f9303-bc0a-43df-b056-cce5ab4f99a6 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:30:29.826689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f9303-bc0a-43df-b056-cce5ab4f99a6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:30:29.826689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f9303-bc0a-43df-b056-cce5ab4f99a6 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:30:29.828094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 753/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 09:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:30:29.833034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453afb25-270a-4e72-87eb-bc4a6fd29f4a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:30:29.833479 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453afb25-270a-4e72-87eb-bc4a6fd29f4a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:30:29.833577 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453afb25-270a-4e72-87eb-bc4a6fd29f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:30:29.922092 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-453afb25-270a-4e72-87eb-bc4a6fd29f4a None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:30:29.923448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-453afb25-270a-4e72-87eb-bc4a6fd29f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:30:30.030117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 759/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 09:30:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:30:30.185697 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f9303-bc0a-43df-b056-cce5ab4f99a6 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:30:30.187861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-452f9303-bc0a-43df-b056-cce5ab4f99a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:30:30.313013 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f9303-bc0a-43df-b056-cce5ab4f99a6 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:30:30.318848 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0993c-3aee-4218-930f-7aa87938df53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:30:30.319459 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0993c-3aee-4218-930f-7aa87938df53 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:30:30.319746 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0993c-3aee-4218-930f-7aa87938df53 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:30:30.320022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0993c-3aee-4218-930f-7aa87938df53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:30:30.330759 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d0993c-3aee-4218-930f-7aa87938df53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:30:30.330759 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d0993c-3aee-4218-930f-7aa87938df53 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:30:30.341031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 754/1513] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:30:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:30:31.190341 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2659c72-5f7c-4ed9-941c-b241f1180c43 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:30:31.190931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2659c72-5f7c-4ed9-941c-b241f1180c43 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:30:31.190931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2659c72-5f7c-4ed9-941c-b241f1180c43 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:30:31.192296 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 760/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 09:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:30:31.197156 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1a67f-6b0a-4f27-92d0-39ce2a9eab0b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:30:31.197636 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1a67f-6b0a-4f27-92d0-39ce2a9eab0b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:30:31.197766 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1a67f-6b0a-4f27-92d0-39ce2a9eab0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:30:31.219305 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b9f1a67f-6b0a-4f27-92d0-39ce2a9eab0b None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:30:31.220508 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b9f1a67f-6b0a-4f27-92d0-39ce2a9eab0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:30:31.228456 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 755/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 09:30:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:30:31.383245 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2659c72-5f7c-4ed9-941c-b241f1180c43 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:30:31.398748 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2659c72-5f7c-4ed9-941c-b241f1180c43 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:30:31.404336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7904e6-721b-4ebe-bd85-ed644b5b3cff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:30:31.404958 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7904e6-721b-4ebe-bd85-ed644b5b3cff service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:30:31.405242 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7904e6-721b-4ebe-bd85-ed644b5b3cff service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:30:31.405515 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7904e6-721b-4ebe-bd85-ed644b5b3cff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:30:31.416190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7904e6-721b-4ebe-bd85-ed644b5b3cff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:30:31.418348 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7904e6-721b-4ebe-bd85-ed644b5b3cff service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:30:31.428146 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 761/1516] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:30:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:30:31.945898 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c303f7-03d1-4f4a-a8e2-65a9aaad1d83 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:30:31.946201 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c303f7-03d1-4f4a-a8e2-65a9aaad1d83 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:30:31.946475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c303f7-03d1-4f4a-a8e2-65a9aaad1d83 None None] PATH_INFO: `/v3` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:30:31.947818 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 756/1517] 10.222.0.16 () {54 vars in 925 bytes} [Fri Sep 2 09:30:31 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:30:31.955971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e41f2-2768-4d3b-8661-0e9ae9deb337 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:30:31.956262 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e41f2-2768-4d3b-8661-0e9ae9deb337 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:30:31.956538 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e41f2-2768-4d3b-8661-0e9ae9deb337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:30:32.018472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-857e41f2-2768-4d3b-8661-0e9ae9deb337 None None] MFA Rules not processed for user `5177fa4d168c4c219e09c7178bbbc790`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:30:32.019736 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-857e41f2-2768-4d3b-8661-0e9ae9deb337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:30:32.038313 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:30:32.099422 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 762/1518] 10.222.0.16 () {58 vars in 1015 bytes} [Fri Sep 2 09:30:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:30:41.214133 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=56 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:30:42.097216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=23 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:30:42.608699 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3edeca-868c-4ac8-b6ab-c95329ca453e None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:30:42.609278 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3edeca-868c-4ac8-b6ab-c95329ca453e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:30:42.609278 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3edeca-868c-4ac8-b6ab-c95329ca453e None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:30:42.610587 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 757/1519] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 09:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:30:42.615549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ee90a-dd87-41cc-9cdb-58d6d5fc9f1c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:30:42.615927 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ee90a-dd87-41cc-9cdb-58d6d5fc9f1c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:30:42.616267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ee90a-dd87-41cc-9cdb-58d6d5fc9f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:30:42.636202 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b80ee90a-dd87-41cc-9cdb-58d6d5fc9f1c None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:30:42.637450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b80ee90a-dd87-41cc-9cdb-58d6d5fc9f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:30:42.645704 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 763/1520] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 09:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:30:42.801261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3edeca-868c-4ac8-b6ab-c95329ca453e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:30:42.807250 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bf9fe-8f1a-491a-814d-2c370187a95d None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:30:42.807602 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bf9fe-8f1a-491a-814d-2c370187a95d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:30:42.807977 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638bf9fe-8f1a-491a-814d-2c370187a95d None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:30:42.809938 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 764/1521] 10.222.0.57 () {54 vars in 961 bytes} [Fri Sep 2 09:30:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:30:42.814756 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e5b31-8264-46e4-8129-1c448a85608e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:30:42.818817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e5b31-8264-46e4-8129-1c448a85608e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:30:42.818817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e5b31-8264-46e4-8129-1c448a85608e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:30:42.819030 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3edeca-868c-4ac8-b6ab-c95329ca453e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:30:42.821757 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db2706-cc35-4742-a660-08a4aebd79cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:30:42.822356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db2706-cc35-4742-a660-08a4aebd79cb service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:30:42.822636 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db2706-cc35-4742-a660-08a4aebd79cb service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:30:42.822941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db2706-cc35-4742-a660-08a4aebd79cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:30:42.831268 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db2706-cc35-4742-a660-08a4aebd79cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:30:42.832907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db2706-cc35-4742-a660-08a4aebd79cb service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:30:42.834529 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-031e5b31-8264-46e4-8129-1c448a85608e None None] MFA Rules not processed for user `5177fa4d168c4c219e09c7178bbbc790`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:30:42.835793 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-031e5b31-8264-46e4-8129-1c448a85608e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:30:42.842930 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 758/1522] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:30:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:30:42.846854 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 765/1523] 10.222.0.57 () {58 vars in 1058 bytes} [Fri Sep 2 09:30:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:30:42.852320 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db2706-cc35-4742-a660-08a4aebd79cb service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:30:42.863990 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db2706-cc35-4742-a660-08a4aebd79cb service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:30:42.869110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e922bf4-be2e-4909-8965-33ba39d5da43 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5177fa4d168c4c219e09c7178bbbc790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:30:42.869702 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e922bf4-be2e-4909-8965-33ba39d5da43 service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:30:42.869978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e922bf4-be2e-4909-8965-33ba39d5da43 service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:30:42.870245 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e922bf4-be2e-4909-8965-33ba39d5da43 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:30:42.879920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e922bf4-be2e-4909-8965-33ba39d5da43 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:30:42.881473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e922bf4-be2e-4909-8965-33ba39d5da43 service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:30:42.891565 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 759/1524] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:30:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:30:43.672670 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9684c60-d5f2-4ecc-802f-77d4c97ba69f None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:30:43.673234 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9684c60-d5f2-4ecc-802f-77d4c97ba69f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:30:43.673234 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9684c60-d5f2-4ecc-802f-77d4c97ba69f None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:30:43.674722 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 766/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Sep 2 09:30:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:30:43.680143 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e0fe4-1fd0-4589-aedf-bcdc8ff8dc61 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:30:43.680143 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e0fe4-1fd0-4589-aedf-bcdc8ff8dc61 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:30:43.680143 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e0fe4-1fd0-4589-aedf-bcdc8ff8dc61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:30:43.700005 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a19e0fe4-1fd0-4589-aedf-bcdc8ff8dc61 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:30:43.701237 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a19e0fe4-1fd0-4589-aedf-bcdc8ff8dc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:30:43.709031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 760/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Sep 2 09:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:30:43.869459 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9684c60-d5f2-4ecc-802f-77d4c97ba69f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:30:43.884169 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9684c60-d5f2-4ecc-802f-77d4c97ba69f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:30:43.889848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944ddf33-4054-436a-ade2-ea3681cf776c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:30:43.890453 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ddf33-4054-436a-ade2-ea3681cf776c service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:30:43.890756 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ddf33-4054-436a-ade2-ea3681cf776c service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:30:43.891034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ddf33-4054-436a-ade2-ea3681cf776c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:30:43.899176 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944ddf33-4054-436a-ade2-ea3681cf776c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:30:43.900749 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944ddf33-4054-436a-ade2-ea3681cf776c service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:30:43.910504 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 767/1527] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:30:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:30:48.278821 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace0a6d-b252-4c4f-adce-60e10bb4c6ed None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:30:48.279334 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace0a6d-b252-4c4f-adce-60e10bb4c6ed None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:30:48.279334 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace0a6d-b252-4c4f-adce-60e10bb4c6ed None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:30:48.280618 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 761/1528] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:30:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:30:48.285688 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302e838-3dff-4c85-95d5-72b9ff8720b7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:30:48.286141 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302e838-3dff-4c85-95d5-72b9ff8720b7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:30:48.286294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302e838-3dff-4c85-95d5-72b9ff8720b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:30:48.305221 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0302e838-3dff-4c85-95d5-72b9ff8720b7 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:30:48.306413 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0302e838-3dff-4c85-95d5-72b9ff8720b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:30:48.314596 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 768/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:30:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:30:48.599201 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace0a6d-b252-4c4f-adce-60e10bb4c6ed None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:30:48.600873 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9ace0a6d-b252-4c4f-adce-60e10bb4c6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:30:48.677238 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ace0a6d-b252-4c4f-adce-60e10bb4c6ed None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:30:48.680756 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab89021-d882-4338-bfb1-e898b834eef6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:30:48.681353 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab89021-d882-4338-bfb1-e898b834eef6 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:30:48.681628 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab89021-d882-4338-bfb1-e898b834eef6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:30:48.681898 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab89021-d882-4338-bfb1-e898b834eef6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:30:48.691130 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab89021-d882-4338-bfb1-e898b834eef6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:30:48.692536 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab89021-d882-4338-bfb1-e898b834eef6 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:30:48.701591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 762/1530] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:30:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:30:50.582064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ad7df-c346-4568-9b18-40f28b4abb06 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:30:50.582602 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ad7df-c346-4568-9b18-40f28b4abb06 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:30:50.582602 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ad7df-c346-4568-9b18-40f28b4abb06 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:30:50.584014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 769/1531] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:30:50.589734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08890df-2d7b-43e9-befd-3ef0124e1eb5 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:30:50.590140 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08890df-2d7b-43e9-befd-3ef0124e1eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:30:50.590272 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08890df-2d7b-43e9-befd-3ef0124e1eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:30:50.609940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c08890df-2d7b-43e9-befd-3ef0124e1eb5 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:30:50.611163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c08890df-2d7b-43e9-befd-3ef0124e1eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:30:50.619243 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 763/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:30:50.907259 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ad7df-c346-4568-9b18-40f28b4abb06 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:30:50.917844 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ad7df-c346-4568-9b18-40f28b4abb06 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:30:50.921337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0f8315-fd1d-4323-92d4-2a40924dfaf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:30:50.921934 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f8315-fd1d-4323-92d4-2a40924dfaf7 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:30:50.922213 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f8315-fd1d-4323-92d4-2a40924dfaf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:30:50.922483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f8315-fd1d-4323-92d4-2a40924dfaf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:30:50.931540 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f8315-fd1d-4323-92d4-2a40924dfaf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:30:50.932957 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f8315-fd1d-4323-92d4-2a40924dfaf7 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:30:50.942161 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 770/1533] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:30:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:30:54.396058 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbab2f9-9d2d-49b7-91ec-fb0266245bd6 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:30:54.396622 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbab2f9-9d2d-49b7-91ec-fb0266245bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:30:54.396622 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbab2f9-9d2d-49b7-91ec-fb0266245bd6 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:30:54.397975 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 764/1534] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:30:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:30:54.403006 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fd3ad-a238-4342-a14c-3673537c48b5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:30:54.403426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fd3ad-a238-4342-a14c-3673537c48b5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:30:54.403549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fd3ad-a238-4342-a14c-3673537c48b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:30:54.422336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b19fd3ad-a238-4342-a14c-3673537c48b5 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:30:54.423555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b19fd3ad-a238-4342-a14c-3673537c48b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:30:54.431247 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 771/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:30:54.989295 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbab2f9-9d2d-49b7-91ec-fb0266245bd6 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:30:54.991002 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-dcbab2f9-9d2d-49b7-91ec-fb0266245bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:30:55.067856 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbab2f9-9d2d-49b7-91ec-fb0266245bd6 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:30:55.071289 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc3bcb-f9ee-4597-aa4b-0bd6b9c0cb34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:30:55.071879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc3bcb-f9ee-4597-aa4b-0bd6b9c0cb34 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:30:55.072157 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc3bcb-f9ee-4597-aa4b-0bd6b9c0cb34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:30:55.072426 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc3bcb-f9ee-4597-aa4b-0bd6b9c0cb34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:30:55.081007 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc3bcb-f9ee-4597-aa4b-0bd6b9c0cb34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:30:55.082438 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc3bcb-f9ee-4597-aa4b-0bd6b9c0cb34 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:30:55.092025 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 765/1536] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:30:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:30:57.003029 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd6dab-2c21-4f1c-aca0-0398727f847a None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:30:57.003556 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd6dab-2c21-4f1c-aca0-0398727f847a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:30:57.003556 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd6dab-2c21-4f1c-aca0-0398727f847a None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:30:57.004841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 772/1537] 127.0.0.1 () {54 vars in 917 bytes} [Fri Sep 2 09:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:30:57.010128 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67275784-5671-4eee-9822-d8ec33ef4887 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:30:57.010522 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67275784-5671-4eee-9822-d8ec33ef4887 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:30:57.010667 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67275784-5671-4eee-9822-d8ec33ef4887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:30:57.029991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-67275784-5671-4eee-9822-d8ec33ef4887 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:30:57.031235 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-67275784-5671-4eee-9822-d8ec33ef4887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:30:57.038897 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 766/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Sep 2 09:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:30:57.592286 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd6dab-2c21-4f1c-aca0-0398727f847a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:30:57.593988 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-47fd6dab-2c21-4f1c-aca0-0398727f847a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:30:57.607469 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd6dab-2c21-4f1c-aca0-0398727f847a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:30:57.610923 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e663d2a0-6ac7-4187-94a7-a12b23895d3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:30:57.611518 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663d2a0-6ac7-4187-94a7-a12b23895d3e service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:30:57.611793 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663d2a0-6ac7-4187-94a7-a12b23895d3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:30:57.612059 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e663d2a0-6ac7-4187-94a7-a12b23895d3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:30:57.621657 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e663d2a0-6ac7-4187-94a7-a12b23895d3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:30:57.623225 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e663d2a0-6ac7-4187-94a7-a12b23895d3e service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:30:57.632393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 773/1539] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:30:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:31:07.025231 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=91 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:31:07.628363 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=30 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:22.057264 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b91747-17e8-4a71-b6be-81b5135bf6f0 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:22.057970 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b91747-17e8-4a71-b6be-81b5135bf6f0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:22.057970 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b91747-17e8-4a71-b6be-81b5135bf6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:22.083794 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-80b91747-17e8-4a71-b6be-81b5135bf6f0 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:22.085220 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-80b91747-17e8-4a71-b6be-81b5135bf6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:22.097267 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 767/1540] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:22.105584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac099cb-4d3b-49b5-9710-fcafa9e6b1e7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:22.106203 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac099cb-4d3b-49b5-9710-fcafa9e6b1e7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:22.106203 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac099cb-4d3b-49b5-9710-fcafa9e6b1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:22.135135 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4ac099cb-4d3b-49b5-9710-fcafa9e6b1e7 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:22.136395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4ac099cb-4d3b-49b5-9710-fcafa9e6b1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:22.150400 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 774/1541] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:22.155655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b91747-17e8-4a71-b6be-81b5135bf6f0 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:22.172985 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b91747-17e8-4a71-b6be-81b5135bf6f0 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:22.180231 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d3bf7-cc21-45d2-b80c-35ce2b6e995f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:22.180431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d3bf7-cc21-45d2-b80c-35ce2b6e995f admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:22.180690 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d3bf7-cc21-45d2-b80c-35ce2b6e995f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:22.180973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d3bf7-cc21-45d2-b80c-35ce2b6e995f admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:22.181642 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d3bf7-cc21-45d2-b80c-35ce2b6e995f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:32:22.181948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d3bf7-cc21-45d2-b80c-35ce2b6e995f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:22.183018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: WARNING py.warnings [None req-533d3bf7-cc21-45d2-b80c-35ce2b6e995f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:32:22.183018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: warnings.warn(msg) Sep 02 09:32:22.183018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: Sep 02 09:32:22.184006 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d3bf7-cc21-45d2-b80c-35ce2b6e995f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:22.206087 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 768/1542] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:32:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:32:22.214746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac099cb-4d3b-49b5-9710-fcafa9e6b1e7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:22.232441 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac099cb-4d3b-49b5-9710-fcafa9e6b1e7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:22.239019 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f1fe2-f267-431b-b1c7-d6eebefbee6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:22.239700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f1fe2-f267-431b-b1c7-d6eebefbee6c admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:22.239981 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f1fe2-f267-431b-b1c7-d6eebefbee6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:22.240251 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f1fe2-f267-431b-b1c7-d6eebefbee6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:22.240890 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5f1fe2-f267-431b-b1c7-d6eebefbee6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:22.242504 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-be5f1fe2-f267-431b-b1c7-d6eebefbee6c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:32:22.242504 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Sep 02 09:32:22.242504 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: Sep 02 09:32:22.243707 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5f1fe2-f267-431b-b1c7-d6eebefbee6c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:22.259030 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcb58d-ce9b-4bf0-bdda-336d3b31f1e4 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:22.259355 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcb58d-ce9b-4bf0-bdda-336d3b31f1e4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:22.259592 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcb58d-ce9b-4bf0-bdda-336d3b31f1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:22.289427 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-19dcb58d-ce9b-4bf0-bdda-336d3b31f1e4 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:22.291019 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-19dcb58d-ce9b-4bf0-bdda-336d3b31f1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:22.306687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 769/1543] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:22.315751 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b81d9d-fb70-4c6d-9425-265a555bb165 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:22.316106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b81d9d-fb70-4c6d-9425-265a555bb165 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:22.316315 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b81d9d-fb70-4c6d-9425-265a555bb165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:22.343356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-50b81d9d-fb70-4c6d-9425-265a555bb165 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:22.344586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-50b81d9d-fb70-4c6d-9425-265a555bb165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:22.363164 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 770/1544] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:32:22.368287 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b81d9d-fb70-4c6d-9425-265a555bb165 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:22.381486 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b81d9d-fb70-4c6d-9425-265a555bb165 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:22.387696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb8879d-a8ad-43cb-8c44-cd32ee7e658d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:22.388323 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8879d-a8ad-43cb-8c44-cd32ee7e658d admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:22.388605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8879d-a8ad-43cb-8c44-cd32ee7e658d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:22.388886 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8879d-a8ad-43cb-8c44-cd32ee7e658d admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:22.389533 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb8879d-a8ad-43cb-8c44-cd32ee7e658d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:32:22.389837 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb8879d-a8ad-43cb-8c44-cd32ee7e658d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:22.391486 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb8879d-a8ad-43cb-8c44-cd32ee7e658d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:22.396250 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 775/1545] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:32:22 2022] POST /identity/v3/projects => generated 383 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:22.401432 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4caac4a-8c52-4772-abd3-32b9e4759388 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:22.401432 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4caac4a-8c52-4772-abd3-32b9e4759388 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:22.401432 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4caac4a-8c52-4772-abd3-32b9e4759388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:22.411643 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 771/1546] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:32:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:32:22.412343 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb8879d-a8ad-43cb-8c44-cd32ee7e658d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:22.433888 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb8879d-a8ad-43cb-8c44-cd32ee7e658d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:22.440374 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e4caac4a-8c52-4772-abd3-32b9e4759388 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:22.443680 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e4caac4a-8c52-4772-abd3-32b9e4759388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:22.526604 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d40eb-8b27-4391-af6a-437f39d9a4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:22.527385 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d40eb-8b27-4391-af6a-437f39d9a4ef admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:22.527788 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d40eb-8b27-4391-af6a-437f39d9a4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:22.528131 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d40eb-8b27-4391-af6a-437f39d9a4ef admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:22.528775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d40eb-8b27-4391-af6a-437f39d9a4ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:22.529853 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: WARNING py.warnings [None req-815d40eb-8b27-4391-af6a-437f39d9a4ef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:32:22.529853 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: warnings.warn(msg) Sep 02 09:32:22.529853 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: Sep 02 09:32:22.531109 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d40eb-8b27-4391-af6a-437f39d9a4ef admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:22.536949 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 776/1547] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:22.537641 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4caac4a-8c52-4772-abd3-32b9e4759388 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:22.547755 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4caac4a-8c52-4772-abd3-32b9e4759388 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:22.552445 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aefbc2-45fd-45a7-842e-010a9462e2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:22.558048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aefbc2-45fd-45a7-842e-010a9462e2a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:22.558048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aefbc2-45fd-45a7-842e-010a9462e2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:22.558048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aefbc2-45fd-45a7-842e-010a9462e2a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:22.558048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aefbc2-45fd-45a7-842e-010a9462e2a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:22.558048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aefbc2-45fd-45a7-842e-010a9462e2a0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:22.643733 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 772/1548] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:32:22 2022] POST /identity/v3/users => generated 292 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:22.644941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30c2de-db1d-4115-a541-e8763d411354 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:22.645229 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30c2de-db1d-4115-a541-e8763d411354 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:22.645505 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b30c2de-db1d-4115-a541-e8763d411354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:22.665556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-9b30c2de-db1d-4115-a541-e8763d411354 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:22.666874 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9b30c2de-db1d-4115-a541-e8763d411354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:22.675085 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 773/1549] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:22.676232 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 777/1550] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:32:22 2022] POST /identity/v3/projects => generated 377 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:22.676439 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8419fe-da01-4608-a194-a14fc9802561 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:22.676858 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8419fe-da01-4608-a194-a14fc9802561 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:22.677141 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8419fe-da01-4608-a194-a14fc9802561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:22.678340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e1cdd-a8b8-4cf5-9ff6-bf1b2913f48f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:22.679334 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e1cdd-a8b8-4cf5-9ff6-bf1b2913f48f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:22.679802 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e1cdd-a8b8-4cf5-9ff6-bf1b2913f48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:22.697541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8a8419fe-da01-4608-a194-a14fc9802561 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:22.701752 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8a8419fe-da01-4608-a194-a14fc9802561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:22.701877 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-db8e1cdd-a8b8-4cf5-9ff6-bf1b2913f48f None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:22.701877 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-db8e1cdd-a8b8-4cf5-9ff6-bf1b2913f48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:22.759518 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 774/1551] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:22.760775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd0551b-93a2-4b95-b91d-6fb375e8199d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:22.761066 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd0551b-93a2-4b95-b91d-6fb375e8199d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:22.761343 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd0551b-93a2-4b95-b91d-6fb375e8199d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:22.775368 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 778/1552] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:22.775368 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92e964-1560-4adc-b59e-c49b844509ae None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:22.775368 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92e964-1560-4adc-b59e-c49b844509ae None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:22.775368 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92e964-1560-4adc-b59e-c49b844509ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:22.781663 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2cd0551b-93a2-4b95-b91d-6fb375e8199d None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:22.782914 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2cd0551b-93a2-4b95-b91d-6fb375e8199d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:22.791639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-cc92e964-1560-4adc-b59e-c49b844509ae None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:22.800520 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 775/1553] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:22.800694 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cc92e964-1560-4adc-b59e-c49b844509ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:22.800794 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b480d-cda9-4881-8765-c4708b65b840 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:22.800794 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b480d-cda9-4881-8765-c4708b65b840 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:22.800794 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b480d-cda9-4881-8765-c4708b65b840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:22.803869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 779/1554] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:22.808569 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92e964-1560-4adc-b59e-c49b844509ae None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:22.817052 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d62b480d-cda9-4881-8765-c4708b65b840 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:22.817989 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92e964-1560-4adc-b59e-c49b844509ae None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:22.818282 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d62b480d-cda9-4881-8765-c4708b65b840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:22.822602 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f920d-d9e5-482f-934d-be486b8983ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:22.823240 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f920d-d9e5-482f-934d-be486b8983ae admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:22.823523 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f920d-d9e5-482f-934d-be486b8983ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:22.823798 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f920d-d9e5-482f-934d-be486b8983ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:22.824448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8f920d-d9e5-482f-934d-be486b8983ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:22.826016 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8f920d-d9e5-482f-934d-be486b8983ae admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:22.826259 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 776/1555] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:22.827089 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b480d-cda9-4881-8765-c4708b65b840 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:22.836890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b480d-cda9-4881-8765-c4708b65b840 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:22.840982 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 780/1556] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:22.845945 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b3679-85b9-4c24-9e0c-5be0cbd65bc8 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:22.846236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b3679-85b9-4c24-9e0c-5be0cbd65bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:22.846543 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b3679-85b9-4c24-9e0c-5be0cbd65bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:22.846754 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318f2f69-8b3e-4dca-a751-5a2fdf9a5c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:22.847378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f2f69-8b3e-4dca-a751-5a2fdf9a5c4e admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:22.847660 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f2f69-8b3e-4dca-a751-5a2fdf9a5c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:22.847927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f2f69-8b3e-4dca-a751-5a2fdf9a5c4e admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:22.848561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318f2f69-8b3e-4dca-a751-5a2fdf9a5c4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:22.850483 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318f2f69-8b3e-4dca-a751-5a2fdf9a5c4e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:22.866903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-113b3679-85b9-4c24-9e0c-5be0cbd65bc8 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:22.873315 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-113b3679-85b9-4c24-9e0c-5be0cbd65bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:22.876329 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 781/1557] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:22.877623 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975f6a6-8021-4318-ad71-9b62d15da363 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:22.877937 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975f6a6-8021-4318-ad71-9b62d15da363 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:22.878213 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1975f6a6-8021-4318-ad71-9b62d15da363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:22.898332 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1975f6a6-8021-4318-ad71-9b62d15da363 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:22.899505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1975f6a6-8021-4318-ad71-9b62d15da363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:22.907413 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 782/1558] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:22.909361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657845b3-acf3-4a1c-8ebb-e730e5dcf7ba None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:22.909680 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657845b3-acf3-4a1c-8ebb-e730e5dcf7ba None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:22.909955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657845b3-acf3-4a1c-8ebb-e730e5dcf7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:22.929476 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-657845b3-acf3-4a1c-8ebb-e730e5dcf7ba None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:22.930766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-657845b3-acf3-4a1c-8ebb-e730e5dcf7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:22.938488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 777/1559] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:32:22 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:22.941743 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 783/1560] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:22.941743 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657845b3-acf3-4a1c-8ebb-e730e5dcf7ba None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:22.941981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5557a8-9df1-4778-912a-1c6e448f84a8 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:22.941981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5557a8-9df1-4778-912a-1c6e448f84a8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:22.941981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5557a8-9df1-4778-912a-1c6e448f84a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:22.946401 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:22.950266 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657845b3-acf3-4a1c-8ebb-e730e5dcf7ba None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:22.955370 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9368199-5cc7-43c8-99ca-a5c0926df1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:22.956111 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9368199-5cc7-43c8-99ca-a5c0926df1d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:22.956396 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9368199-5cc7-43c8-99ca-a5c0926df1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:22.956661 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:22.956774 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9368199-5cc7-43c8-99ca-a5c0926df1d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:22.957316 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9368199-5cc7-43c8-99ca-a5c0926df1d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:32:22.957684 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9368199-5cc7-43c8-99ca-a5c0926df1d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:22.959348 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9368199-5cc7-43c8-99ca-a5c0926df1d4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:22.962799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ce5557a8-9df1-4778-912a-1c6e448f84a8 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:22.964070 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ce5557a8-9df1-4778-912a-1c6e448f84a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:22.971891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 784/1561] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:32:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:32:22.972047 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 778/1562] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:22.972867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5557a8-9df1-4778-912a-1c6e448f84a8 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:22.973295 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac258fc-9dba-46f4-a0cf-e7fb7032ece6 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:22.973528 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac258fc-9dba-46f4-a0cf-e7fb7032ece6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:22.973801 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac258fc-9dba-46f4-a0cf-e7fb7032ece6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:22.985049 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5557a8-9df1-4778-912a-1c6e448f84a8 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:22.988210 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c00a9-238c-4590-b13d-68f5e51829d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:22.988871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c00a9-238c-4590-b13d-68f5e51829d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:22.989150 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c00a9-238c-4590-b13d-68f5e51829d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:22.989424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c00a9-238c-4590-b13d-68f5e51829d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:22.990059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c00a9-238c-4590-b13d-68f5e51829d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:22.991703 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683c00a9-238c-4590-b13d-68f5e51829d1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:22.997133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3ac258fc-9dba-46f4-a0cf-e7fb7032ece6 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:22.999660 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3ac258fc-9dba-46f4-a0cf-e7fb7032ece6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:23.002577 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 779/1563] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:23.006524 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c00a9-238c-4590-b13d-68f5e51829d1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:23.007983 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 785/1564] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:23.008790 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac258fc-9dba-46f4-a0cf-e7fb7032ece6 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:23.016654 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c00a9-238c-4590-b13d-68f5e51829d1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:23.018432 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac258fc-9dba-46f4-a0cf-e7fb7032ece6 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:23.021284 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034f440-53e2-423f-a25e-a6e18819e064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:23.021930 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034f440-53e2-423f-a25e-a6e18819e064 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:23.022220 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034f440-53e2-423f-a25e-a6e18819e064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:23.023045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2034f440-53e2-423f-a25e-a6e18819e064 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:23.023534 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52fd9c5-989e-43b3-9add-6ee67f1a7447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:23.023690 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2034f440-53e2-423f-a25e-a6e18819e064 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:32:23.023975 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2034f440-53e2-423f-a25e-a6e18819e064 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:23.024099 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52fd9c5-989e-43b3-9add-6ee67f1a7447 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:23.024382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52fd9c5-989e-43b3-9add-6ee67f1a7447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:23.024658 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52fd9c5-989e-43b3-9add-6ee67f1a7447 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:23.025295 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52fd9c5-989e-43b3-9add-6ee67f1a7447 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:32:23.025595 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52fd9c5-989e-43b3-9add-6ee67f1a7447 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:23.025698 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2034f440-53e2-423f-a25e-a6e18819e064 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:23.027233 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52fd9c5-989e-43b3-9add-6ee67f1a7447 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:23.050868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 780/1565] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:32:23.050868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034f440-53e2-423f-a25e-a6e18819e064 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:23.051189 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 786/1566] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:32:23.051189 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52fd9c5-989e-43b3-9add-6ee67f1a7447 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:23.051189 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52fd9c5-989e-43b3-9add-6ee67f1a7447 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:23.055836 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ecbc8-86e4-4517-bcf9-9536706b3bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:23.056180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2034f440-53e2-423f-a25e-a6e18819e064 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:23.056500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ecbc8-86e4-4517-bcf9-9536706b3bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:23.056796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ecbc8-86e4-4517-bcf9-9536706b3bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:23.057082 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ecbc8-86e4-4517-bcf9-9536706b3bc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:23.057721 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ecbc8-86e4-4517-bcf9-9536706b3bc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:32:23.058030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ecbc8-86e4-4517-bcf9-9536706b3bc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:23.059671 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ecbc8-86e4-4517-bcf9-9536706b3bc4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:23.061406 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4fe9c3-35a9-467e-abae-3c146c55be59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:23.062018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4fe9c3-35a9-467e-abae-3c146c55be59 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:23.062303 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4fe9c3-35a9-467e-abae-3c146c55be59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:23.062570 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4fe9c3-35a9-467e-abae-3c146c55be59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:23.063229 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4fe9c3-35a9-467e-abae-3c146c55be59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:23.064843 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4fe9c3-35a9-467e-abae-3c146c55be59 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:23.073917 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 787/1567] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:32:23.081182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ecbc8-86e4-4517-bcf9-9536706b3bc4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:23.081318 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 781/1568] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:23.081318 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4fe9c3-35a9-467e-abae-3c146c55be59 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:23.090472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4fe9c3-35a9-467e-abae-3c146c55be59 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:23.090868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ecbc8-86e4-4517-bcf9-9536706b3bc4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:23.095092 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304dc1f7-c5bc-4dea-8bd6-41addb5c3c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:23.095431 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d78ba-d0ee-4d7f-925f-fbd430065bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:23.095687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304dc1f7-c5bc-4dea-8bd6-41addb5c3c59 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:23.095971 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304dc1f7-c5bc-4dea-8bd6-41addb5c3c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:23.096076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d78ba-d0ee-4d7f-925f-fbd430065bee admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:23.096247 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304dc1f7-c5bc-4dea-8bd6-41addb5c3c59 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:23.096351 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d78ba-d0ee-4d7f-925f-fbd430065bee admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:23.096548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d78ba-d0ee-4d7f-925f-fbd430065bee admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:23.096888 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304dc1f7-c5bc-4dea-8bd6-41addb5c3c59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:32:23.097194 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304dc1f7-c5bc-4dea-8bd6-41addb5c3c59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:23.097463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581d78ba-d0ee-4d7f-925f-fbd430065bee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:23.098827 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304dc1f7-c5bc-4dea-8bd6-41addb5c3c59 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:23.099460 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581d78ba-d0ee-4d7f-925f-fbd430065bee admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:23.111413 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 782/1569] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:32:23.112222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304dc1f7-c5bc-4dea-8bd6-41addb5c3c59 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:23.121687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304dc1f7-c5bc-4dea-8bd6-41addb5c3c59 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:23.126274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262c4fc-9023-4cf8-ad4d-1eb36393d75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:23.126947 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262c4fc-9023-4cf8-ad4d-1eb36393d75b admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:23.127228 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262c4fc-9023-4cf8-ad4d-1eb36393d75b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:23.127504 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262c4fc-9023-4cf8-ad4d-1eb36393d75b admin admin] PATH_INFO: `/v3/projects/62d5c9f1ea8b4929860a24a3189e0379/users/9637e0dfaee84bbc8700a9951f3ba756/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:23.142276 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2262c4fc-9023-4cf8-ad4d-1eb36393d75b admin admin] RBAC: Authorizing `identity:create_grant(project_id=62d5c9f1ea8b4929860a24a3189e0379, user_id=9637e0dfaee84bbc8700a9951f3ba756, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:23.144903 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2262c4fc-9023-4cf8-ad4d-1eb36393d75b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:23.202373 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 788/1570] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:32:23 2022] POST /identity/v3/projects => generated 405 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:23.202373 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d78ba-d0ee-4d7f-925f-fbd430065bee admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:23.222585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 783/1571] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:23 2022] PUT /identity/v3/projects/62d5c9f1ea8b4929860a24a3189e0379/users/9637e0dfaee84bbc8700a9951f3ba756/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:23.236966 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262c4fc-9023-4cf8-ad4d-1eb36393d75b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:23.260952 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d78ba-d0ee-4d7f-925f-fbd430065bee admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:23.294083 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2262c4fc-9023-4cf8-ad4d-1eb36393d75b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:23.298952 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947b72e-4a0a-4008-99c4-6c8f55a2892f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:23.299560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947b72e-4a0a-4008-99c4-6c8f55a2892f admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:23.299961 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947b72e-4a0a-4008-99c4-6c8f55a2892f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:23.300245 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947b72e-4a0a-4008-99c4-6c8f55a2892f admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:23.300908 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947b72e-4a0a-4008-99c4-6c8f55a2892f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:23.302870 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947b72e-4a0a-4008-99c4-6c8f55a2892f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:23.316552 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ff494-9ba0-4f35-9228-1dcf6125ca19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:23.317168 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ff494-9ba0-4f35-9228-1dcf6125ca19 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:23.317443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ff494-9ba0-4f35-9228-1dcf6125ca19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:23.317714 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ff494-9ba0-4f35-9228-1dcf6125ca19 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:23.318359 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ff494-9ba0-4f35-9228-1dcf6125ca19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:32:23.318760 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ff494-9ba0-4f35-9228-1dcf6125ca19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:23.320844 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ff494-9ba0-4f35-9228-1dcf6125ca19 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:23.332046 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 789/1572] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:32:23.332869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ff494-9ba0-4f35-9228-1dcf6125ca19 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:23.342123 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ff494-9ba0-4f35-9228-1dcf6125ca19 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:23.346601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8980b-6d5a-47b8-9b90-b91eaf75bde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:23.347215 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8980b-6d5a-47b8-9b90-b91eaf75bde6 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:23.347498 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8980b-6d5a-47b8-9b90-b91eaf75bde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:23.347804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec8980b-6d5a-47b8-9b90-b91eaf75bde6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:23.348426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec8980b-6d5a-47b8-9b90-b91eaf75bde6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:23.351045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec8980b-6d5a-47b8-9b90-b91eaf75bde6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:23.398548 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 784/1573] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:32:23 2022] POST /identity/v3/projects => generated 385 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:23.398548 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947b72e-4a0a-4008-99c4-6c8f55a2892f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:23.446676 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 790/1574] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:32:23 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:23.447760 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8980b-6d5a-47b8-9b90-b91eaf75bde6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:23.456328 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947b72e-4a0a-4008-99c4-6c8f55a2892f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:23.465503 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:23.473525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:23.511936 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec8980b-6d5a-47b8-9b90-b91eaf75bde6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:23.514080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aca490-932e-4d69-ba91-42139bf48321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:23.515072 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aca490-932e-4d69-ba91-42139bf48321 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:23.515402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aca490-932e-4d69-ba91-42139bf48321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:23.515676 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aca490-932e-4d69-ba91-42139bf48321 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:23.516331 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aca490-932e-4d69-ba91-42139bf48321 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:23.516877 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e879a-1c7c-481f-b39a-2f29f5c12636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:23.517522 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e879a-1c7c-481f-b39a-2f29f5c12636 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:23.517805 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e879a-1c7c-481f-b39a-2f29f5c12636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:23.517909 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aca490-932e-4d69-ba91-42139bf48321 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:23.518078 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30e879a-1c7c-481f-b39a-2f29f5c12636 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:23.518741 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e879a-1c7c-481f-b39a-2f29f5c12636 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:23.520622 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30e879a-1c7c-481f-b39a-2f29f5c12636 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:23.528053 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 785/1575] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:23 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:23.528846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aca490-932e-4d69-ba91-42139bf48321 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:23.538838 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aca490-932e-4d69-ba91-42139bf48321 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:23.543614 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d5761-7efe-4c2c-b8e2-e277ab7e378c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:23.544203 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d5761-7efe-4c2c-b8e2-e277ab7e378c admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:23.544484 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d5761-7efe-4c2c-b8e2-e277ab7e378c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:23.544759 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d5761-7efe-4c2c-b8e2-e277ab7e378c admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:23.545374 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d5761-7efe-4c2c-b8e2-e277ab7e378c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:23.547431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d5761-7efe-4c2c-b8e2-e277ab7e378c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:23.589793 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 791/1576] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:32:23 2022] POST /identity/v3/projects => generated 371 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:23.590771 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e879a-1c7c-481f-b39a-2f29f5c12636 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:23.624115 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 786/1577] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:32:23 2022] POST /identity/v3/projects => generated 379 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:23.626982 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d89c3-c620-42cd-b152-8e467b6e2063 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:23.627274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d89c3-c620-42cd-b152-8e467b6e2063 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:23.627769 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d89c3-c620-42cd-b152-8e467b6e2063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:23.656101 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30e879a-1c7c-481f-b39a-2f29f5c12636 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:23.656322 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3a4d89c3-c620-42cd-b152-8e467b6e2063 None None] MFA Rules not processed for user `9637e0dfaee84bbc8700a9951f3ba756`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:23.657442 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3a4d89c3-c620-42cd-b152-8e467b6e2063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:23.713791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cf897-a6d3-4431-a3a6-f950e387b2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:23.714318 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cf897-a6d3-4431-a3a6-f950e387b2f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:23.714596 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cf897-a6d3-4431-a3a6-f950e387b2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:23.714900 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cf897-a6d3-4431-a3a6-f950e387b2f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:23.715528 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832cf897-a6d3-4431-a3a6-f950e387b2f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:23.717470 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832cf897-a6d3-4431-a3a6-f950e387b2f6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:23.724977 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 787/1578] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:23.726393 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d89c3-c620-42cd-b152-8e467b6e2063 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:23.736473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d89c3-c620-42cd-b152-8e467b6e2063 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:23.740902 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b320cae8-b5fb-4640-8a08-37268a3d326e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:23.741514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320cae8-b5fb-4640-8a08-37268a3d326e admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:23.741789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320cae8-b5fb-4640-8a08-37268a3d326e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:23.742863 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320cae8-b5fb-4640-8a08-37268a3d326e admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:23.742863 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b320cae8-b5fb-4640-8a08-37268a3d326e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:23.744606 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b320cae8-b5fb-4640-8a08-37268a3d326e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:23.824971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 792/1579] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:32:23 2022] POST /identity/v3/users => generated 303 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:23.824971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cf897-a6d3-4431-a3a6-f950e387b2f6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:23.832526 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cf897-a6d3-4431-a3a6-f950e387b2f6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:23.833532 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 788/1580] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:32:23 2022] POST /identity/v3/projects => generated 393 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:23.835754 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b320cae8-b5fb-4640-8a08-37268a3d326e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:23.875625 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:23.899813 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ef4e5-889d-4d6d-978a-d4c63a446a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:23.900361 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b320cae8-b5fb-4640-8a08-37268a3d326e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:23.900509 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ef4e5-889d-4d6d-978a-d4c63a446a39 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:23.900703 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ef4e5-889d-4d6d-978a-d4c63a446a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:23.900974 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04ef4e5-889d-4d6d-978a-d4c63a446a39 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:23.901602 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ef4e5-889d-4d6d-978a-d4c63a446a39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:23.903562 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04ef4e5-889d-4d6d-978a-d4c63a446a39 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:23.904950 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9539af6b-9ea2-4674-9447-7efcac35578a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:23.905550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539af6b-9ea2-4674-9447-7efcac35578a admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:23.905826 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539af6b-9ea2-4674-9447-7efcac35578a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:23.906100 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539af6b-9ea2-4674-9447-7efcac35578a admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:23.906731 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9539af6b-9ea2-4674-9447-7efcac35578a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:23.908655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9539af6b-9ea2-4674-9447-7efcac35578a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:23.917488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:24.020689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 793/1581] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:32:23 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:24.021430 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ef4e5-889d-4d6d-978a-d4c63a446a39 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:24.031647 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04ef4e5-889d-4d6d-978a-d4c63a446a39 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:24.036153 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa9229-87d6-4a69-bb17-7d1bc90165f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:24.036795 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa9229-87d6-4a69-bb17-7d1bc90165f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:24.037072 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa9229-87d6-4a69-bb17-7d1bc90165f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:24.037342 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa9229-87d6-4a69-bb17-7d1bc90165f8 admin admin] PATH_INFO: `/v3/projects/ea33fe1eaf1047f78312a8e9fa9152c4/users/f65429de28ae41578c51a6b9099fea5c/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:24.045063 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 789/1582] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:32:23 2022] POST /identity/v3/users => generated 286 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:24.045063 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9539af6b-9ea2-4674-9447-7efcac35578a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:24.054208 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9539af6b-9ea2-4674-9447-7efcac35578a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:24.054393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa9229-87d6-4a69-bb17-7d1bc90165f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea33fe1eaf1047f78312a8e9fa9152c4, user_id=f65429de28ae41578c51a6b9099fea5c, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:24.055472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-a5fa9229-87d6-4a69-bb17-7d1bc90165f8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:32:24.055472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Sep 02 09:32:24.055472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: Sep 02 09:32:24.057368 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa9229-87d6-4a69-bb17-7d1bc90165f8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:24.059112 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15de16e-6ae6-457c-a2c7-861056de54f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:24.059714 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15de16e-6ae6-457c-a2c7-861056de54f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:24.060124 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15de16e-6ae6-457c-a2c7-861056de54f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:24.060409 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15de16e-6ae6-457c-a2c7-861056de54f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:24.061081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15de16e-6ae6-457c-a2c7-861056de54f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:24.063004 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15de16e-6ae6-457c-a2c7-861056de54f7 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:24.160780 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 794/1583] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:24 2022] PUT /identity/v3/projects/ea33fe1eaf1047f78312a8e9fa9152c4/users/f65429de28ae41578c51a6b9099fea5c/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:24.164563 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 790/1584] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:32:24 2022] POST /identity/v3/users => generated 286 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:24.164763 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa9229-87d6-4a69-bb17-7d1bc90165f8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:24.164857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15de16e-6ae6-457c-a2c7-861056de54f7 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:24.177770 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15de16e-6ae6-457c-a2c7-861056de54f7 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:24.232946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa9229-87d6-4a69-bb17-7d1bc90165f8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:24.235549 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa0193-c6be-43c0-9363-0bdd861a6540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:24.236161 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa0193-c6be-43c0-9363-0bdd861a6540 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:24.236437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa0193-c6be-43c0-9363-0bdd861a6540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:24.236709 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa0193-c6be-43c0-9363-0bdd861a6540 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:24.237326 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaa0193-c6be-43c0-9363-0bdd861a6540 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:24.237833 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137f569-b01d-4050-8309-fecb482e9b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:24.238444 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137f569-b01d-4050-8309-fecb482e9b63 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:24.238747 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137f569-b01d-4050-8309-fecb482e9b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:24.239027 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137f569-b01d-4050-8309-fecb482e9b63 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:24.239235 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaa0193-c6be-43c0-9363-0bdd861a6540 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:24.239651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b137f569-b01d-4050-8309-fecb482e9b63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:24.242120 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b137f569-b01d-4050-8309-fecb482e9b63 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:24.375067 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 795/1585] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:32:24 2022] POST /identity/v3/users => generated 290 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:24.375911 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137f569-b01d-4050-8309-fecb482e9b63 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:24.386362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137f569-b01d-4050-8309-fecb482e9b63 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:24.391206 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20640942-a09f-46c9-8d49-d39e75fdd7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:24.391797 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20640942-a09f-46c9-8d49-d39e75fdd7a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:24.392076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20640942-a09f-46c9-8d49-d39e75fdd7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:24.392385 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20640942-a09f-46c9-8d49-d39e75fdd7a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:24.393025 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20640942-a09f-46c9-8d49-d39e75fdd7a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:24.394590 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20640942-a09f-46c9-8d49-d39e75fdd7a5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:24.398735 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 791/1586] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:32:24 2022] POST /identity/v3/projects => generated 383 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:24.399667 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa0193-c6be-43c0-9363-0bdd861a6540 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:24.407788 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 796/1587] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:24.407788 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20640942-a09f-46c9-8d49-d39e75fdd7a5 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:24.415171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20640942-a09f-46c9-8d49-d39e75fdd7a5 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:24.465529 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa0193-c6be-43c0-9363-0bdd861a6540 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:24.470231 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b668257-d187-46b4-8df1-386ff9700d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:24.470865 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b668257-d187-46b4-8df1-386ff9700d55 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:24.471148 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b668257-d187-46b4-8df1-386ff9700d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:24.471424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b668257-d187-46b4-8df1-386ff9700d55 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:24.472053 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b668257-d187-46b4-8df1-386ff9700d55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:24.472416 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae7ebe-cfa7-4ef2-ae70-e6fcfd8e1eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:24.473020 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae7ebe-cfa7-4ef2-ae70-e6fcfd8e1eb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:24.473298 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae7ebe-cfa7-4ef2-ae70-e6fcfd8e1eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:24.473569 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae7ebe-cfa7-4ef2-ae70-e6fcfd8e1eb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:24.473959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b668257-d187-46b4-8df1-386ff9700d55 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:24.474205 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae7ebe-cfa7-4ef2-ae70-e6fcfd8e1eb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:24.475833 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ae7ebe-cfa7-4ef2-ae70-e6fcfd8e1eb0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:24.485494 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 797/1588] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:24.486278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae7ebe-cfa7-4ef2-ae70-e6fcfd8e1eb0 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:24.498238 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae7ebe-cfa7-4ef2-ae70-e6fcfd8e1eb0 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:24.502838 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced15de3-075b-4714-aef4-3a1daa36c8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:24.503422 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced15de3-075b-4714-aef4-3a1daa36c8f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:24.503700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced15de3-075b-4714-aef4-3a1daa36c8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:24.503971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced15de3-075b-4714-aef4-3a1daa36c8f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:24.504597 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced15de3-075b-4714-aef4-3a1daa36c8f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:24.506172 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced15de3-075b-4714-aef4-3a1daa36c8f1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:24.517210 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 798/1589] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:24.518315 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfc7de-a0b4-4d48-9d81-32f3b44c81a7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:24.518594 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfc7de-a0b4-4d48-9d81-32f3b44c81a7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:24.518889 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dfc7de-a0b4-4d48-9d81-32f3b44c81a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:24.531220 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:24.537166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-61dfc7de-a0b4-4d48-9d81-32f3b44c81a7 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:24.538924 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-61dfc7de-a0b4-4d48-9d81-32f3b44c81a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:24.546610 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 799/1590] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:24.547397 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dfc7de-a0b4-4d48-9d81-32f3b44c81a7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:24.557017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dfc7de-a0b4-4d48-9d81-32f3b44c81a7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:24.561675 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f5850-04a1-4de9-ba84-95b7de8c68db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:24.562276 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f5850-04a1-4de9-ba84-95b7de8c68db admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:24.562556 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f5850-04a1-4de9-ba84-95b7de8c68db admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:24.562866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56f5850-04a1-4de9-ba84-95b7de8c68db admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:24.563517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56f5850-04a1-4de9-ba84-95b7de8c68db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:24.565138 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56f5850-04a1-4de9-ba84-95b7de8c68db admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:24.576002 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 800/1591] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:24.576832 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f5850-04a1-4de9-ba84-95b7de8c68db admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:24.586957 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56f5850-04a1-4de9-ba84-95b7de8c68db admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:24.591627 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4429d-d92d-474c-a0f2-17d2976a9a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:24.592204 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4429d-d92d-474c-a0f2-17d2976a9a1b admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:24.592486 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4429d-d92d-474c-a0f2-17d2976a9a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:24.592760 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a4429d-d92d-474c-a0f2-17d2976a9a1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:24.593390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a4429d-d92d-474c-a0f2-17d2976a9a1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:24.594997 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a4429d-d92d-474c-a0f2-17d2976a9a1b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:24.604440 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 801/1592] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:24.605179 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4429d-d92d-474c-a0f2-17d2976a9a1b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:24.614350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a4429d-d92d-474c-a0f2-17d2976a9a1b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:24.616091 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 792/1593] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:32:24 2022] POST /identity/v3/users => generated 297 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:24.616091 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b668257-d187-46b4-8df1-386ff9700d55 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:24.619434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66092ab3-7f9b-4a8d-98c4-9e9b0d69ddb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:24.620021 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66092ab3-7f9b-4a8d-98c4-9e9b0d69ddb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:24.620306 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66092ab3-7f9b-4a8d-98c4-9e9b0d69ddb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:24.620581 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66092ab3-7f9b-4a8d-98c4-9e9b0d69ddb9 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:24.621202 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66092ab3-7f9b-4a8d-98c4-9e9b0d69ddb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:24.623132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66092ab3-7f9b-4a8d-98c4-9e9b0d69ddb9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:24.625988 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b668257-d187-46b4-8df1-386ff9700d55 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:24.630633 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168f1a3-1fa7-4ad7-b54c-1c27fe961668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:24.631266 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168f1a3-1fa7-4ad7-b54c-1c27fe961668 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:24.631551 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168f1a3-1fa7-4ad7-b54c-1c27fe961668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:24.632351 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168f1a3-1fa7-4ad7-b54c-1c27fe961668 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:24.633266 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168f1a3-1fa7-4ad7-b54c-1c27fe961668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:24.635016 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168f1a3-1fa7-4ad7-b54c-1c27fe961668 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:24.645392 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 793/1594] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:24.646194 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168f1a3-1fa7-4ad7-b54c-1c27fe961668 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:24.656281 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168f1a3-1fa7-4ad7-b54c-1c27fe961668 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:24.661046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79748781-99d3-4d4f-966e-8795ee49a103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:24.661639 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79748781-99d3-4d4f-966e-8795ee49a103 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:24.661902 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79748781-99d3-4d4f-966e-8795ee49a103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:24.662174 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79748781-99d3-4d4f-966e-8795ee49a103 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:24.662830 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79748781-99d3-4d4f-966e-8795ee49a103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:24.664448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79748781-99d3-4d4f-966e-8795ee49a103 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:24.674696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 794/1595] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:24.675475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79748781-99d3-4d4f-966e-8795ee49a103 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:24.684659 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79748781-99d3-4d4f-966e-8795ee49a103 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:24.689889 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557f35c-9c5d-477d-8143-f38875fb6ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:24.689889 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557f35c-9c5d-477d-8143-f38875fb6ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:24.690438 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557f35c-9c5d-477d-8143-f38875fb6ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:24.690551 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4557f35c-9c5d-477d-8143-f38875fb6ec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:24.691223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557f35c-9c5d-477d-8143-f38875fb6ec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:24.692773 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4557f35c-9c5d-477d-8143-f38875fb6ec4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:24.697183 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 802/1596] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:32:24 2022] POST /identity/v3/users => generated 298 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:24.698122 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66092ab3-7f9b-4a8d-98c4-9e9b0d69ddb9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:24.708493 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 795/1597] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:24.708493 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557f35c-9c5d-477d-8143-f38875fb6ec4 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:24.716448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4557f35c-9c5d-477d-8143-f38875fb6ec4 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:24.721458 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6246794-a755-4870-97a4-548fcc79e36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:24.722043 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6246794-a755-4870-97a4-548fcc79e36c admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:24.722328 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6246794-a755-4870-97a4-548fcc79e36c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:24.722606 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6246794-a755-4870-97a4-548fcc79e36c admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:24.723278 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6246794-a755-4870-97a4-548fcc79e36c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:24.724847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6246794-a755-4870-97a4-548fcc79e36c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:24.735076 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 796/1598] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:24.735854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6246794-a755-4870-97a4-548fcc79e36c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:24.743816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:INSERT=9,keystone:SELECT=91 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:24.747620 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6246794-a755-4870-97a4-548fcc79e36c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:24.752502 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce079192-d985-4fcd-9a19-11a6391a319e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:24.753771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce079192-d985-4fcd-9a19-11a6391a319e admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:24.753771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce079192-d985-4fcd-9a19-11a6391a319e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:24.753771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce079192-d985-4fcd-9a19-11a6391a319e admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:24.754314 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce079192-d985-4fcd-9a19-11a6391a319e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:24.754539 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66092ab3-7f9b-4a8d-98c4-9e9b0d69ddb9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:24.756036 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce079192-d985-4fcd-9a19-11a6391a319e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:24.759112 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048349c-e786-4f26-8212-7abb115ae861 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:24.759738 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048349c-e786-4f26-8212-7abb115ae861 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:24.760004 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048349c-e786-4f26-8212-7abb115ae861 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:24.760278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048349c-e786-4f26-8212-7abb115ae861 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:24.771862 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 797/1599] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:24.773666 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce079192-d985-4fcd-9a19-11a6391a319e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:24.776117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048349c-e786-4f26-8212-7abb115ae861 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:24.777675 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048349c-e786-4f26-8212-7abb115ae861 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:24.784662 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce079192-d985-4fcd-9a19-11a6391a319e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:24.787859 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 803/1600] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:24.788866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048349c-e786-4f26-8212-7abb115ae861 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:24.789723 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f489997-dd0a-40e1-a9dd-fa975fed97e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:24.790430 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f489997-dd0a-40e1-a9dd-fa975fed97e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:24.790820 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f489997-dd0a-40e1-a9dd-fa975fed97e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:24.791238 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f489997-dd0a-40e1-a9dd-fa975fed97e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:24.791965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f489997-dd0a-40e1-a9dd-fa975fed97e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:24.793628 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f489997-dd0a-40e1-a9dd-fa975fed97e9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:24.798849 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048349c-e786-4f26-8212-7abb115ae861 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:24.804135 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa381c80-72d9-4905-95ba-2c631b414dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:24.804868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa381c80-72d9-4905-95ba-2c631b414dc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:24.805246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa381c80-72d9-4905-95ba-2c631b414dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:24.805616 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa381c80-72d9-4905-95ba-2c631b414dc4 admin admin] PATH_INFO: `/v3/projects/5dc2dc3eab8d4594a4566a4220870b8f/users/f50514c32784400fb6db57217481e2fa/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:24.806401 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 798/1601] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:24.810251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f489997-dd0a-40e1-a9dd-fa975fed97e9 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:24.822608 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f489997-dd0a-40e1-a9dd-fa975fed97e9 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:24.823281 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa381c80-72d9-4905-95ba-2c631b414dc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dc2dc3eab8d4594a4566a4220870b8f, user_id=f50514c32784400fb6db57217481e2fa, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:24.825630 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa381c80-72d9-4905-95ba-2c631b414dc4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:24.828382 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e7032-55c6-4e26-b42a-a641a7fdc803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:24.829099 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e7032-55c6-4e26-b42a-a641a7fdc803 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:24.829386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e7032-55c6-4e26-b42a-a641a7fdc803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:24.829665 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e7032-55c6-4e26-b42a-a641a7fdc803 admin admin] PATH_INFO: `/v3/projects/2f0e11fc747c4619a370655d8b489cd4/users/9d0680d3e3644bb4a52e140a12f5f54d/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:24.848027 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4e7032-55c6-4e26-b42a-a641a7fdc803 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f0e11fc747c4619a370655d8b489cd4, user_id=9d0680d3e3644bb4a52e140a12f5f54d, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:24.850806 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4e7032-55c6-4e26-b42a-a641a7fdc803 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:24.932203 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 799/1602] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:24 2022] PUT /identity/v3/projects/2f0e11fc747c4619a370655d8b489cd4/users/9d0680d3e3644bb4a52e140a12f5f54d/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:24.932412 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 804/1603] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:24 2022] PUT /identity/v3/projects/5dc2dc3eab8d4594a4566a4220870b8f/users/f50514c32784400fb6db57217481e2fa/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:24.932412 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa381c80-72d9-4905-95ba-2c631b414dc4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:24.932562 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e7032-55c6-4e26-b42a-a641a7fdc803 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:25.005987 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e7032-55c6-4e26-b42a-a641a7fdc803 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:25.007883 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa381c80-72d9-4905-95ba-2c631b414dc4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:25.011614 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90060fca-e81f-439d-8a12-fbcce610bccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:25.015448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90060fca-e81f-439d-8a12-fbcce610bccf admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:25.015448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90060fca-e81f-439d-8a12-fbcce610bccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:25.015448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90060fca-e81f-439d-8a12-fbcce610bccf admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:25.015756 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f949e264-8123-4cd7-aa9e-0d9670bae0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:25.015869 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90060fca-e81f-439d-8a12-fbcce610bccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:25.015973 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949e264-8123-4cd7-aa9e-0d9670bae0aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:25.015973 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949e264-8123-4cd7-aa9e-0d9670bae0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:25.015973 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949e264-8123-4cd7-aa9e-0d9670bae0aa admin admin] PATH_INFO: `/v3/projects/7f784a9048784529aaf1febe6fe1322d/users/4d632542a6e24e84ab479f1ac1eed112/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:25.016233 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90060fca-e81f-439d-8a12-fbcce610bccf admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:25.025311 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 800/1604] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:25.026094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90060fca-e81f-439d-8a12-fbcce610bccf admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:25.034301 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f949e264-8123-4cd7-aa9e-0d9670bae0aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f784a9048784529aaf1febe6fe1322d, user_id=4d632542a6e24e84ab479f1ac1eed112, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:25.035368 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90060fca-e81f-439d-8a12-fbcce610bccf admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:25.036982 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f949e264-8123-4cd7-aa9e-0d9670bae0aa admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:25.040033 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dafb63-f907-4d05-8ae2-b866101d93ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:25.040665 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dafb63-f907-4d05-8ae2-b866101d93ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:25.041056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dafb63-f907-4d05-8ae2-b866101d93ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:25.041332 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dafb63-f907-4d05-8ae2-b866101d93ad admin admin] PATH_INFO: `/v3/projects/0f0b7885fff048ca80407f436430e768/users/8aa60f0d2bac47a5836d0773680b39c9/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:25.056194 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dafb63-f907-4d05-8ae2-b866101d93ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f0b7885fff048ca80407f436430e768, user_id=8aa60f0d2bac47a5836d0773680b39c9, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:25.058857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dafb63-f907-4d05-8ae2-b866101d93ad admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:25.107326 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 805/1605] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:24 2022] PUT /identity/v3/projects/7f784a9048784529aaf1febe6fe1322d/users/4d632542a6e24e84ab479f1ac1eed112/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:25.107880 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f949e264-8123-4cd7-aa9e-0d9670bae0aa admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:25.155732 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:25.168269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 801/1606] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:25 2022] PUT /identity/v3/projects/0f0b7885fff048ca80407f436430e768/users/8aa60f0d2bac47a5836d0773680b39c9/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:25.168269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dafb63-f907-4d05-8ae2-b866101d93ad admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:25.196204 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f949e264-8123-4cd7-aa9e-0d9670bae0aa admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:25.238270 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dafb63-f907-4d05-8ae2-b866101d93ad admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:25.243738 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d237c3-fd19-47a2-b9f1-c76c1f2c9ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:25.256475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d237c3-fd19-47a2-b9f1-c76c1f2c9ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:25.256475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d237c3-fd19-47a2-b9f1-c76c1f2c9ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:25.256475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d237c3-fd19-47a2-b9f1-c76c1f2c9ed2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:25.256475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d237c3-fd19-47a2-b9f1-c76c1f2c9ed2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:25.256475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d237c3-fd19-47a2-b9f1-c76c1f2c9ed2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:25.261256 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 802/1607] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:25 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:25.261256 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ff832-fd6d-43dc-ab15-e2dee46a1f92 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:25.261256 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ff832-fd6d-43dc-ab15-e2dee46a1f92 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:25.261256 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ff832-fd6d-43dc-ab15-e2dee46a1f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:25.262384 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab618a-1536-472c-ba59-057718ccf2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:25.263066 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab618a-1536-472c-ba59-057718ccf2af admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:25.263348 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab618a-1536-472c-ba59-057718ccf2af admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:25.263623 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab618a-1536-472c-ba59-057718ccf2af admin admin] PATH_INFO: `/v3/projects/e553fe4257cc4983ac82027b05a53e65/users/d4d4f092761d44a2a30f0e1d044888b6/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:25.278614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ab618a-1536-472c-ba59-057718ccf2af admin admin] RBAC: Authorizing `identity:create_grant(project_id=e553fe4257cc4983ac82027b05a53e65, user_id=d4d4f092761d44a2a30f0e1d044888b6, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:25.281300 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ab618a-1536-472c-ba59-057718ccf2af admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:25.287683 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-302ff832-fd6d-43dc-ab15-e2dee46a1f92 None None] MFA Rules not processed for user `f50514c32784400fb6db57217481e2fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:25.289484 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-302ff832-fd6d-43dc-ab15-e2dee46a1f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:25.301802 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:25.353541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 803/1608] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:25 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:25.354891 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e676c-e800-4fc5-9c0c-5a2993399fae None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:25.355282 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e676c-e800-4fc5-9c0c-5a2993399fae None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:25.355661 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e676c-e800-4fc5-9c0c-5a2993399fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:25.384958 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-648e676c-e800-4fc5-9c0c-5a2993399fae None None] MFA Rules not processed for user `9d0680d3e3644bb4a52e140a12f5f54d`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:25.386188 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-648e676c-e800-4fc5-9c0c-5a2993399fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:25.415059 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 806/1609] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:25 2022] PUT /identity/v3/projects/e553fe4257cc4983ac82027b05a53e65/users/d4d4f092761d44a2a30f0e1d044888b6/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:25.415059 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab618a-1536-472c-ba59-057718ccf2af admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:25.458226 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 804/1610] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:25.459530 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71b91c-cec1-4563-a68e-a047c407e4cb None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:25.459910 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71b91c-cec1-4563-a68e-a047c407e4cb None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:25.460655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71b91c-cec1-4563-a68e-a047c407e4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:25.490436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab618a-1536-472c-ba59-057718ccf2af admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:25.496066 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493aa6b6-93f8-4fb1-b60b-05bdbaeb5600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:25.497003 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493aa6b6-93f8-4fb1-b60b-05bdbaeb5600 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:25.497604 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493aa6b6-93f8-4fb1-b60b-05bdbaeb5600 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:25.498037 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493aa6b6-93f8-4fb1-b60b-05bdbaeb5600 admin admin] PATH_INFO: `/v3/projects/e2650310037e4858852a02b8cf19e7bd/users/e548f129bdc04a3bbc2ed7618d742b08/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:25.499233 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2c71b91c-cec1-4563-a68e-a047c407e4cb None None] MFA Rules not processed for user `4d632542a6e24e84ab479f1ac1eed112`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:25.500631 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2c71b91c-cec1-4563-a68e-a047c407e4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:25.527493 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493aa6b6-93f8-4fb1-b60b-05bdbaeb5600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2650310037e4858852a02b8cf19e7bd, user_id=e548f129bdc04a3bbc2ed7618d742b08, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:25.530277 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493aa6b6-93f8-4fb1-b60b-05bdbaeb5600 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:25.576139 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 805/1611] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:25.577634 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0a261-1f85-4cd8-a355-546a2926df8f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:25.577933 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0a261-1f85-4cd8-a355-546a2926df8f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:25.578206 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0a261-1f85-4cd8-a355-546a2926df8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:25.607282 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4fa0a261-1f85-4cd8-a355-546a2926df8f None None] MFA Rules not processed for user `8aa60f0d2bac47a5836d0773680b39c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:25.608486 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4fa0a261-1f85-4cd8-a355-546a2926df8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:25.617768 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 807/1612] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:25 2022] PUT /identity/v3/projects/e2650310037e4858852a02b8cf19e7bd/users/e548f129bdc04a3bbc2ed7618d742b08/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:25.619534 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493aa6b6-93f8-4fb1-b60b-05bdbaeb5600 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:25.638939 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:25.666467 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:25.683528 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 806/1613] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:25.688395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0a261-1f85-4cd8-a355-546a2926df8f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:25.689862 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493aa6b6-93f8-4fb1-b60b-05bdbaeb5600 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:25.695518 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08201fec-f448-4282-bacd-0c5df6e57c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:25.696191 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08201fec-f448-4282-bacd-0c5df6e57c8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:25.696465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08201fec-f448-4282-bacd-0c5df6e57c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:25.696749 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08201fec-f448-4282-bacd-0c5df6e57c8b admin admin] PATH_INFO: `/v3/projects/c2945ae512e14bfbb7d02dedc4de7cd4/users/c79aaf1f72634eefb5e89c8005afe54b/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:25.701027 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0a261-1f85-4cd8-a355-546a2926df8f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:25.705813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9727d0-9977-4fea-bb42-f6adeaeec3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:25.706420 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9727d0-9977-4fea-bb42-f6adeaeec3a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:25.709436 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9727d0-9977-4fea-bb42-f6adeaeec3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:25.709436 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9727d0-9977-4fea-bb42-f6adeaeec3a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:25.709436 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9727d0-9977-4fea-bb42-f6adeaeec3a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:25.709893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9727d0-9977-4fea-bb42-f6adeaeec3a9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:25.712197 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08201fec-f448-4282-bacd-0c5df6e57c8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2945ae512e14bfbb7d02dedc4de7cd4, user_id=c79aaf1f72634eefb5e89c8005afe54b, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:25.714989 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08201fec-f448-4282-bacd-0c5df6e57c8b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:25.863114 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 808/1614] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:25 2022] PUT /identity/v3/projects/c2945ae512e14bfbb7d02dedc4de7cd4/users/c79aaf1f72634eefb5e89c8005afe54b/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:25.864457 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33108a7e-ed0c-477f-baa8-d9ddcdddaee8 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:25.864739 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33108a7e-ed0c-477f-baa8-d9ddcdddaee8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:25.865008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33108a7e-ed0c-477f-baa8-d9ddcdddaee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:25.884834 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-33108a7e-ed0c-477f-baa8-d9ddcdddaee8 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:25.886112 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-33108a7e-ed0c-477f-baa8-d9ddcdddaee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:25.891470 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 807/1615] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:32:25 2022] POST /identity/v3/projects => generated 403 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:25.892952 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553bd890-f27f-44a2-bbd0-5ceaac99efdb None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:25.893313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553bd890-f27f-44a2-bbd0-5ceaac99efdb None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:25.893676 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553bd890-f27f-44a2-bbd0-5ceaac99efdb None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:25.895277 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 808/1616] 10.222.0.57 () {54 vars in 962 bytes} [Fri Sep 2 09:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:32:25.896723 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd50aaf-b493-4f2f-a364-a818a4321e7e None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:25.897004 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd50aaf-b493-4f2f-a364-a818a4321e7e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:25.897275 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd50aaf-b493-4f2f-a364-a818a4321e7e None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:25.898554 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 809/1617] 10.222.0.57 () {54 vars in 962 bytes} [Fri Sep 2 09:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:32:25.899279 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd50aaf-b493-4f2f-a364-a818a4321e7e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:25.953560 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 809/1618] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:25.954791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33108a7e-ed0c-477f-baa8-d9ddcdddaee8 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:25.965034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33108a7e-ed0c-477f-baa8-d9ddcdddaee8 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:25.977612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd50aaf-b493-4f2f-a364-a818a4321e7e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:25.983156 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3139b5e3-e2ec-4fd3-86a6-31827c4c018c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:25.984352 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139b5e3-e2ec-4fd3-86a6-31827c4c018c admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:25.984352 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139b5e3-e2ec-4fd3-86a6-31827c4c018c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:25.984747 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3139b5e3-e2ec-4fd3-86a6-31827c4c018c admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:25.985485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3139b5e3-e2ec-4fd3-86a6-31827c4c018c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:25.987224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3139b5e3-e2ec-4fd3-86a6-31827c4c018c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:25.999358 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 810/1619] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:25 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:26.000890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f33db-78aa-4276-8ccf-318a809e2b79 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:26.001270 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f33db-78aa-4276-8ccf-318a809e2b79 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:26.001998 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f33db-78aa-4276-8ccf-318a809e2b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:26.025755 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-9e3f33db-78aa-4276-8ccf-318a809e2b79 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:26.027039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9e3f33db-78aa-4276-8ccf-318a809e2b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:26.029394 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39502eef-a19d-4379-a6c3-e673f1f06f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:26.030006 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39502eef-a19d-4379-a6c3-e673f1f06f32 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:26.030284 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39502eef-a19d-4379-a6c3-e673f1f06f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:26.030739 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39502eef-a19d-4379-a6c3-e673f1f06f32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:26.032825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39502eef-a19d-4379-a6c3-e673f1f06f32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:26.033477 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39502eef-a19d-4379-a6c3-e673f1f06f32 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:26.034748 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 811/1620] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:26.035513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f33db-78aa-4276-8ccf-318a809e2b79 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:26.045330 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f33db-78aa-4276-8ccf-318a809e2b79 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:26.049906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a08597-9163-4f1c-bacb-a4002e448429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:26.050495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a08597-9163-4f1c-bacb-a4002e448429 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:26.050934 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a08597-9163-4f1c-bacb-a4002e448429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:26.051210 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a08597-9163-4f1c-bacb-a4002e448429 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:26.051840 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a08597-9163-4f1c-bacb-a4002e448429 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:26.054299 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a08597-9163-4f1c-bacb-a4002e448429 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:26.164063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 810/1621] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:32:25 2022] POST /identity/v3/projects => generated 371 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:26.165121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d6b57-84b3-4484-9886-b5f931dd4484 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:26.165469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 812/1622] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:32:26 2022] POST /identity/v3/users => generated 308 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:26.165588 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d6b57-84b3-4484-9886-b5f931dd4484 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:26.165852 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1d6b57-84b3-4484-9886-b5f931dd4484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:26.166934 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c60b1-6ff3-4798-998d-31d5660a9a56 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:26.167233 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c60b1-6ff3-4798-998d-31d5660a9a56 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:26.167682 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c60b1-6ff3-4798-998d-31d5660a9a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:26.209024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-bf1d6b57-84b3-4484-9886-b5f931dd4484 None None] MFA Rules not processed for user `1da355c8be2d496bb08feb0fe74e04b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:26.210379 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bf1d6b57-84b3-4484-9886-b5f931dd4484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:26.211145 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e72c60b1-6ff3-4798-998d-31d5660a9a56 None None] MFA Rules not processed for user `1da355c8be2d496bb08feb0fe74e04b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:26.212540 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e72c60b1-6ff3-4798-998d-31d5660a9a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:26.289100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 811/1623] 10.222.0.57 () {58 vars in 1059 bytes} [Fri Sep 2 09:32:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:26.294343 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1d6b57-84b3-4484-9886-b5f931dd4484 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:26.294343 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bf1d6b57-84b3-4484-9886-b5f931dd4484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:26.294562 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:26.310861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 813/1624] 10.222.0.57 () {58 vars in 1059 bytes} [Fri Sep 2 09:32:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:26.313393 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c60b1-6ff3-4798-998d-31d5660a9a56 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:26.340300 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:26.359428 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1d6b57-84b3-4484-9886-b5f931dd4484 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:26.364250 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e04fb-6850-44d4-b3c1-16bcafe411c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:26.365095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e04fb-6850-44d4-b3c1-16bcafe411c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:26.365380 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e04fb-6850-44d4-b3c1-16bcafe411c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:26.365654 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e04fb-6850-44d4-b3c1-16bcafe411c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:26.381779 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c60b1-6ff3-4798-998d-31d5660a9a56 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:26.388694 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f5d2d-1ccb-4fa1-a021-fc06a7570236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:26.389903 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f5d2d-1ccb-4fa1-a021-fc06a7570236 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:26.390210 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f5d2d-1ccb-4fa1-a021-fc06a7570236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:26.390606 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f5d2d-1ccb-4fa1-a021-fc06a7570236 admin admin] PATH_INFO: `/v3/projects/e2650310037e4858852a02b8cf19e7bd/users/e548f129bdc04a3bbc2ed7618d742b08/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:26.393586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f5d2d-1ccb-4fa1-a021-fc06a7570236 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2650310037e4858852a02b8cf19e7bd, user_id=e548f129bdc04a3bbc2ed7618d742b08, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:26.397724 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f5d2d-1ccb-4fa1-a021-fc06a7570236 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:26.415756 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 814/1625] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:26 2022] PUT /identity/v3/projects/e2650310037e4858852a02b8cf19e7bd/users/e548f129bdc04a3bbc2ed7618d742b08/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:26.416729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f5d2d-1ccb-4fa1-a021-fc06a7570236 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:26.450991 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996e04fb-6850-44d4-b3c1-16bcafe411c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:26.452512 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996e04fb-6850-44d4-b3c1-16bcafe411c0 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:26.466036 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 812/1626] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:26.468799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e04fb-6850-44d4-b3c1-16bcafe411c0 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:26.482248 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e04fb-6850-44d4-b3c1-16bcafe411c0 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:26.490813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f5d2d-1ccb-4fa1-a021-fc06a7570236 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:26.495898 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28b6fae-cda7-4dbf-a26b-7912dfe16e30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:26.496509 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b6fae-cda7-4dbf-a26b-7912dfe16e30 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:26.497040 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b6fae-cda7-4dbf-a26b-7912dfe16e30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:26.497040 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b6fae-cda7-4dbf-a26b-7912dfe16e30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:26.567151 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b24681-9b9b-47a7-bce9-ebb00e180faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:26.567791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b24681-9b9b-47a7-bce9-ebb00e180faa admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:26.568440 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b24681-9b9b-47a7-bce9-ebb00e180faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:26.576856 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b24681-9b9b-47a7-bce9-ebb00e180faa admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:26.577677 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b24681-9b9b-47a7-bce9-ebb00e180faa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:26.579869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b24681-9b9b-47a7-bce9-ebb00e180faa admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:26.587037 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28b6fae-cda7-4dbf-a26b-7912dfe16e30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:26.588771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28b6fae-cda7-4dbf-a26b-7912dfe16e30 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:26.603641 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 815/1627] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:26.604799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28b6fae-cda7-4dbf-a26b-7912dfe16e30 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:26.621510 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28b6fae-cda7-4dbf-a26b-7912dfe16e30 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:26.627158 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a3f7b-0576-49ec-ba60-27a3fbabd435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:26.627782 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a3f7b-0576-49ec-ba60-27a3fbabd435 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:26.628060 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a3f7b-0576-49ec-ba60-27a3fbabd435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:26.628331 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a3f7b-0576-49ec-ba60-27a3fbabd435 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:26.628972 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1a3f7b-0576-49ec-ba60-27a3fbabd435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:26.630647 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1a3f7b-0576-49ec-ba60-27a3fbabd435 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:26.644703 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 816/1628] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:26.645467 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a3f7b-0576-49ec-ba60-27a3fbabd435 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:26.703191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:26.706931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 813/1629] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:32:26 2022] POST /identity/v3/users => generated 292 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:26.709549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b24681-9b9b-47a7-bce9-ebb00e180faa admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:26.737749 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a3f7b-0576-49ec-ba60-27a3fbabd435 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:26.743414 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301614d0-d551-40c6-9b2b-6d8e4c10914c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:26.744027 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301614d0-d551-40c6-9b2b-6d8e4c10914c service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:26.744306 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301614d0-d551-40c6-9b2b-6d8e4c10914c service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:26.744579 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301614d0-d551-40c6-9b2b-6d8e4c10914c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:26.786532 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b24681-9b9b-47a7-bce9-ebb00e180faa admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:26.792703 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56b344-c843-456d-a3a6-53c37d5b787d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:26.793318 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56b344-c843-456d-a3a6-53c37d5b787d service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:26.793756 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56b344-c843-456d-a3a6-53c37d5b787d service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:26.794037 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f56b344-c843-456d-a3a6-53c37d5b787d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:26.816430 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301614d0-d551-40c6-9b2b-6d8e4c10914c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:26.817814 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:26.818061 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301614d0-d551-40c6-9b2b-6d8e4c10914c service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:26.831551 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 817/1630] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:26.833435 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d640072-8b6f-4c52-960b-c03b05b1c2b3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:26.833868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d640072-8b6f-4c52-960b-c03b05b1c2b3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:26.834511 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d640072-8b6f-4c52-960b-c03b05b1c2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:26.866965 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f56b344-c843-456d-a3a6-53c37d5b787d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:26.868758 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f56b344-c843-456d-a3a6-53c37d5b787d service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:26.886566 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0d640072-8b6f-4c52-960b-c03b05b1c2b3 None None] MFA Rules not processed for user `e548f129bdc04a3bbc2ed7618d742b08`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:26.888235 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0d640072-8b6f-4c52-960b-c03b05b1c2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:26.888797 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 814/1631] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:26.905487 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56b344-c843-456d-a3a6-53c37d5b787d service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:26.929547 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f56b344-c843-456d-a3a6-53c37d5b787d service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:26.942301 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7f25e-6501-4d65-98cb-6febfc4fe5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:26.946024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7f25e-6501-4d65-98cb-6febfc4fe5b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:26.946024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7f25e-6501-4d65-98cb-6febfc4fe5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:26.946024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c7f25e-6501-4d65-98cb-6febfc4fe5b6 admin admin] PATH_INFO: `/v3/projects/7b2548e96bcf4b3b9ac9ad8cbd662efc/users/6d74dfccb6fb47bd8265f6b7dceb25b0/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:26.989722 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c7f25e-6501-4d65-98cb-6febfc4fe5b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b2548e96bcf4b3b9ac9ad8cbd662efc, user_id=6d74dfccb6fb47bd8265f6b7dceb25b0, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:26.999161 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c7f25e-6501-4d65-98cb-6febfc4fe5b6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:27.030611 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 818/1632] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:27.032151 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d640072-8b6f-4c52-960b-c03b05b1c2b3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:27.050563 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d640072-8b6f-4c52-960b-c03b05b1c2b3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:27.060205 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4e4bc-95f0-4584-abf5-12010f85dd6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:27.064351 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4e4bc-95f0-4584-abf5-12010f85dd6b admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:27.064351 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4e4bc-95f0-4584-abf5-12010f85dd6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:27.064351 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4e4bc-95f0-4584-abf5-12010f85dd6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:27.064351 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4e4bc-95f0-4584-abf5-12010f85dd6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:27.064351 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4e4bc-95f0-4584-abf5-12010f85dd6b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:27.089386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 819/1633] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:27.097337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 815/1634] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:26 2022] PUT /identity/v3/projects/7b2548e96bcf4b3b9ac9ad8cbd662efc/users/6d74dfccb6fb47bd8265f6b7dceb25b0/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:27.097337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7f25e-6501-4d65-98cb-6febfc4fe5b6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:27.097547 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4e4bc-95f0-4584-abf5-12010f85dd6b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:27.192463 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:27.210334 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4e4bc-95f0-4584-abf5-12010f85dd6b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:27.214064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c7f25e-6501-4d65-98cb-6febfc4fe5b6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:27.222075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb0198-e11b-40d8-bf8a-c01b4c473912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:27.222075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb0198-e11b-40d8-bf8a-c01b4c473912 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:27.222075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb0198-e11b-40d8-bf8a-c01b4c473912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:27.222075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb0198-e11b-40d8-bf8a-c01b4c473912 admin admin] PATH_INFO: `/v3/projects/991f0a2854874d87805ab613b34fc7d9/users/cc2cdf8654d747c8bed881317eda55d5/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:27.236988 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb0198-e11b-40d8-bf8a-c01b4c473912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=991f0a2854874d87805ab613b34fc7d9, user_id=cc2cdf8654d747c8bed881317eda55d5, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:27.240078 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cb0198-e11b-40d8-bf8a-c01b4c473912 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:27.270503 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d44a00-ab4e-444d-b480-f0f7a80db1c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:27.271530 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d44a00-ab4e-444d-b480-f0f7a80db1c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:27.271910 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d44a00-ab4e-444d-b480-f0f7a80db1c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:27.272360 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d44a00-ab4e-444d-b480-f0f7a80db1c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:27.338263 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d44a00-ab4e-444d-b480-f0f7a80db1c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:27.339910 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d44a00-ab4e-444d-b480-f0f7a80db1c5 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:27.341344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:27.348193 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 816/1635] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:27 2022] PUT /identity/v3/projects/991f0a2854874d87805ab613b34fc7d9/users/cc2cdf8654d747c8bed881317eda55d5/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:27.349024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb0198-e11b-40d8-bf8a-c01b4c473912 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:27.353640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 820/1636] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:27 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:27.355124 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba0cff-0a69-4a74-ab7e-a69822219214 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:27.355478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba0cff-0a69-4a74-ab7e-a69822219214 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:27.355800 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba0cff-0a69-4a74-ab7e-a69822219214 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:27.357628 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 821/1637] 10.222.0.57 () {54 vars in 912 bytes} [Fri Sep 2 09:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:32:27.358803 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1d6bf-d466-4b09-91bc-669f093af399 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:27.359155 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1d6bf-d466-4b09-91bc-669f093af399 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:27.359507 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e1d6bf-d466-4b09-91bc-669f093af399 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:27.361112 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 822/1638] 10.222.0.57 () {54 vars in 912 bytes} [Fri Sep 2 09:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:32:27.362252 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1d6bf-d466-4b09-91bc-669f093af399 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:27.493099 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e1d6bf-d466-4b09-91bc-669f093af399 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:27.508989 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa19d1bc-221b-405b-8116-5e0aeb2f85a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:27.516714 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19d1bc-221b-405b-8116-5e0aeb2f85a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:27.516714 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19d1bc-221b-405b-8116-5e0aeb2f85a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:27.516714 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19d1bc-221b-405b-8116-5e0aeb2f85a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:27.516714 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa19d1bc-221b-405b-8116-5e0aeb2f85a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:27.516714 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa19d1bc-221b-405b-8116-5e0aeb2f85a1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:27.518338 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cb0198-e11b-40d8-bf8a-c01b4c473912 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:27.525804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de54943-31d4-4034-a1dc-f4910c78dc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:27.526681 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de54943-31d4-4034-a1dc-f4910c78dc8c admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:27.527066 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de54943-31d4-4034-a1dc-f4910c78dc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:27.527408 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de54943-31d4-4034-a1dc-f4910c78dc8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:27.530951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 823/1639] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:27.531121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de54943-31d4-4034-a1dc-f4910c78dc8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:27.531121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de54943-31d4-4034-a1dc-f4910c78dc8c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:27.545624 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 817/1640] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:27.547213 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dc43c-ded1-4a9d-a9ce-5e05a30fc9e7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:27.547354 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cdb27d-1f70-4979-b49d-85c50f87d11f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:27.547500 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cdb27d-1f70-4979-b49d-85c50f87d11f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:27.547767 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cdb27d-1f70-4979-b49d-85c50f87d11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:27.547890 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dc43c-ded1-4a9d-a9ce-5e05a30fc9e7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:27.548214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61dc43c-ded1-4a9d-a9ce-5e05a30fc9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:27.573814 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e61dc43c-ded1-4a9d-a9ce-5e05a30fc9e7 None None] MFA Rules not processed for user `1da355c8be2d496bb08feb0fe74e04b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:27.575199 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e61dc43c-ded1-4a9d-a9ce-5e05a30fc9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:27.588591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a3cdb27d-1f70-4979-b49d-85c50f87d11f None None] MFA Rules not processed for user `1da355c8be2d496bb08feb0fe74e04b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:27.589990 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a3cdb27d-1f70-4979-b49d-85c50f87d11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:27.641895 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:27.663214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 818/1641] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Sep 2 09:32:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:27.685619 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61dc43c-ded1-4a9d-a9ce-5e05a30fc9e7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:27.818205 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 824/1642] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Sep 2 09:32:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:27.818205 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cdb27d-1f70-4979-b49d-85c50f87d11f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:27.818205 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cdb27d-1f70-4979-b49d-85c50f87d11f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:27.818205 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971a3b8-0902-446b-97ee-1b6cfb80967f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:27.819598 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971a3b8-0902-446b-97ee-1b6cfb80967f admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:27.819598 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971a3b8-0902-446b-97ee-1b6cfb80967f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:27.819598 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971a3b8-0902-446b-97ee-1b6cfb80967f admin admin] PATH_INFO: `/v3/projects/991f0a2854874d87805ab613b34fc7d9/users/cc2cdf8654d747c8bed881317eda55d5/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:27.819598 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5971a3b8-0902-446b-97ee-1b6cfb80967f admin admin] RBAC: Authorizing `identity:create_grant(project_id=991f0a2854874d87805ab613b34fc7d9, user_id=cc2cdf8654d747c8bed881317eda55d5, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:27.819598 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5971a3b8-0902-446b-97ee-1b6cfb80967f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:27.820253 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61dc43c-ded1-4a9d-a9ce-5e05a30fc9e7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:27.820351 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 825/1643] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:27 2022] PUT /identity/v3/projects/991f0a2854874d87805ab613b34fc7d9/users/cc2cdf8654d747c8bed881317eda55d5/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:27.820351 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971a3b8-0902-446b-97ee-1b6cfb80967f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:27.904385 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90bc500-64d9-4fa4-8793-a712433e30bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:27.906834 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bc500-64d9-4fa4-8793-a712433e30bd service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:27.908528 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bc500-64d9-4fa4-8793-a712433e30bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:27.909611 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90bc500-64d9-4fa4-8793-a712433e30bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:27.948053 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:27.951457 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5971a3b8-0902-446b-97ee-1b6cfb80967f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:27.963364 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ec523-b594-48a3-a005-0ab1ca99cd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:27.964056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ec523-b594-48a3-a005-0ab1ca99cd8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:27.964337 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ec523-b594-48a3-a005-0ab1ca99cd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:27.964612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323ec523-b594-48a3-a005-0ab1ca99cd8e admin admin] PATH_INFO: `/v3/projects/7b2548e96bcf4b3b9ac9ad8cbd662efc/users/6d74dfccb6fb47bd8265f6b7dceb25b0/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:27.967718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ec523-b594-48a3-a005-0ab1ca99cd8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b2548e96bcf4b3b9ac9ad8cbd662efc, user_id=6d74dfccb6fb47bd8265f6b7dceb25b0, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:27.970395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323ec523-b594-48a3-a005-0ab1ca99cd8e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:27.986492 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 826/1644] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:27 2022] PUT /identity/v3/projects/7b2548e96bcf4b3b9ac9ad8cbd662efc/users/6d74dfccb6fb47bd8265f6b7dceb25b0/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:27.987406 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ec523-b594-48a3-a005-0ab1ca99cd8e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:28.019925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90bc500-64d9-4fa4-8793-a712433e30bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:28.022846 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90bc500-64d9-4fa4-8793-a712433e30bd service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:28.039042 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 819/1645] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:28.040388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a2c47f-6584-4cb8-98a7-46854a63a800 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:28.040745 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a2c47f-6584-4cb8-98a7-46854a63a800 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:28.041033 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a2c47f-6584-4cb8-98a7-46854a63a800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:28.081060 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-02a2c47f-6584-4cb8-98a7-46854a63a800 None None] MFA Rules not processed for user `cc2cdf8654d747c8bed881317eda55d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:28.082694 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-02a2c47f-6584-4cb8-98a7-46854a63a800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:28.084118 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323ec523-b594-48a3-a005-0ab1ca99cd8e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:28.088029 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9267010-1974-41ef-8dd0-110fda683155 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:28.092344 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9267010-1974-41ef-8dd0-110fda683155 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:28.092344 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9267010-1974-41ef-8dd0-110fda683155 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:28.092344 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9267010-1974-41ef-8dd0-110fda683155 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:28.173752 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 820/1646] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:28 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:28.175320 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b0a39-4243-4ea5-bd21-86894725600d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:28.175614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b0a39-4243-4ea5-bd21-86894725600d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:28.176032 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b0a39-4243-4ea5-bd21-86894725600d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:28.199233 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:28.204720 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9267010-1974-41ef-8dd0-110fda683155 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:28.208394 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9267010-1974-41ef-8dd0-110fda683155 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:28.210623 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5e1b0a39-4243-4ea5-bd21-86894725600d None None] MFA Rules not processed for user `6d74dfccb6fb47bd8265f6b7dceb25b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:28.212759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5e1b0a39-4243-4ea5-bd21-86894725600d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:28.237026 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 827/1647] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:28.350547 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 821/1648] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:28 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:29.440658 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9267010-1974-41ef-8dd0-110fda683155 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:29.453082 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9267010-1974-41ef-8dd0-110fda683155 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:29.457417 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05215d53-eba3-4172-8620-32114df52066 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:29.458053 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05215d53-eba3-4172-8620-32114df52066 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:29.458338 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05215d53-eba3-4172-8620-32114df52066 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:29.458615 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05215d53-eba3-4172-8620-32114df52066 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:29.471494 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05215d53-eba3-4172-8620-32114df52066 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:29.473089 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05215d53-eba3-4172-8620-32114df52066 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:29.486799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 828/1649] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:29.680040 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b0a39-4243-4ea5-bd21-86894725600d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:29.695025 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b0a39-4243-4ea5-bd21-86894725600d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:29.705535 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09c8ed-0bf1-4fbe-80ab-61b18dea590e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:29.705535 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09c8ed-0bf1-4fbe-80ab-61b18dea590e service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:29.705778 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09c8ed-0bf1-4fbe-80ab-61b18dea590e service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:29.706050 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09c8ed-0bf1-4fbe-80ab-61b18dea590e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:29.718495 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:29.727617 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09c8ed-0bf1-4fbe-80ab-61b18dea590e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:29.729058 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09c8ed-0bf1-4fbe-80ab-61b18dea590e service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:29.751169 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 822/1650] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:29 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:29.926556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05215d53-eba3-4172-8620-32114df52066 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:29.938385 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05215d53-eba3-4172-8620-32114df52066 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:29.953269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68def9-eb6b-483f-a81a-09f8d0a7afbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:29.953855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68def9-eb6b-483f-a81a-09f8d0a7afbb service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:29.954138 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68def9-eb6b-483f-a81a-09f8d0a7afbb service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:29.954407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68def9-eb6b-483f-a81a-09f8d0a7afbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:29.979710 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68def9-eb6b-483f-a81a-09f8d0a7afbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:29.981470 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68def9-eb6b-483f-a81a-09f8d0a7afbb service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:30.004255 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 829/1651] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:29 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:30.750793 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09c8ed-0bf1-4fbe-80ab-61b18dea590e service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:30.930455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09c8ed-0bf1-4fbe-80ab-61b18dea590e service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:30.934652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e996d43-2226-431f-88b8-6a8393e383dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:30.935509 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e996d43-2226-431f-88b8-6a8393e383dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:30.935807 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e996d43-2226-431f-88b8-6a8393e383dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:30.936085 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e996d43-2226-431f-88b8-6a8393e383dc admin admin] PATH_INFO: `/v3/users/f50514c32784400fb6db57217481e2fa` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:30.937414 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e996d43-2226-431f-88b8-6a8393e383dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f50514c32784400fb6db57217481e2fa)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:30.938508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: WARNING py.warnings [None req-3e996d43-2226-431f-88b8-6a8393e383dc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:32:30.938508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: warnings.warn(msg) Sep 02 09:32:30.938508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: Sep 02 09:32:30.939793 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e996d43-2226-431f-88b8-6a8393e383dc admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:31.300124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3e996d43-2226-431f-88b8-6a8393e383dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f50514c32784400fb6db57217481e2fa'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:31.393961 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3e996d43-2226-431f-88b8-6a8393e383dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f50514c32784400fb6db57217481e2fa'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:31.412933 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3e996d43-2226-431f-88b8-6a8393e383dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f50514c32784400fb6db57217481e2fa'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:31.412933 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3e996d43-2226-431f-88b8-6a8393e383dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f50514c32784400fb6db57217481e2fa'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:31.444184 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3e996d43-2226-431f-88b8-6a8393e383dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f50514c32784400fb6db57217481e2fa'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:31.486158 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 823/1652] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:32:30 2022] DELETE /identity/v3/users/f50514c32784400fb6db57217481e2fa => generated 0 bytes in 736 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:31.492737 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ce4ed-a49e-4c39-a723-e42034897a31 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:31.499051 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ce4ed-a49e-4c39-a723-e42034897a31 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:31.499051 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ce4ed-a49e-4c39-a723-e42034897a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:31.547746 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-fc4ce4ed-a49e-4c39-a723-e42034897a31 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:31.549214 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fc4ce4ed-a49e-4c39-a723-e42034897a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:31.677137 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 830/1653] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:31.751072 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e996d43-2226-431f-88b8-6a8393e383dc admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:31.759175 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3e996d43-2226-431f-88b8-6a8393e383dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:31.972414 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e996d43-2226-431f-88b8-6a8393e383dc admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:31.979210 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbfd21-6941-46b3-821c-f341ee5ded32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:31.980525 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbfd21-6941-46b3-821c-f341ee5ded32 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:31.981726 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbfd21-6941-46b3-821c-f341ee5ded32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:31.982200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbfd21-6941-46b3-821c-f341ee5ded32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:32.007815 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbfd21-6941-46b3-821c-f341ee5ded32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:32.011568 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbfd21-6941-46b3-821c-f341ee5ded32 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:32.027368 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ce4ed-a49e-4c39-a723-e42034897a31 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:32.029359 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fc4ce4ed-a49e-4c39-a723-e42034897a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:32.034337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 824/1654] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:32.057651 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ce4ed-a49e-4c39-a723-e42034897a31 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:32.067300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc3cc26-89eb-4672-bf55-22973a9f1825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:32.067920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3cc26-89eb-4672-bf55-22973a9f1825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:32.068262 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3cc26-89eb-4672-bf55-22973a9f1825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:32.068536 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3cc26-89eb-4672-bf55-22973a9f1825 admin admin] PATH_INFO: `/v3/users/8aa60f0d2bac47a5836d0773680b39c9` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:32.072866 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc3cc26-89eb-4672-bf55-22973a9f1825 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8aa60f0d2bac47a5836d0773680b39c9)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:32.083004 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc3cc26-89eb-4672-bf55-22973a9f1825 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:32.515263 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4fc3cc26-89eb-4672-bf55-22973a9f1825 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8aa60f0d2bac47a5836d0773680b39c9'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:32.556288 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4fc3cc26-89eb-4672-bf55-22973a9f1825 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8aa60f0d2bac47a5836d0773680b39c9'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:32.656955 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4fc3cc26-89eb-4672-bf55-22973a9f1825 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8aa60f0d2bac47a5836d0773680b39c9'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:32.657615 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4fc3cc26-89eb-4672-bf55-22973a9f1825 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8aa60f0d2bac47a5836d0773680b39c9'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:32.691979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4fc3cc26-89eb-4672-bf55-22973a9f1825 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8aa60f0d2bac47a5836d0773680b39c9'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:32.723672 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 831/1655] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:32:32 2022] DELETE /identity/v3/users/8aa60f0d2bac47a5836d0773680b39c9 => generated 0 bytes in 697 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:32.730304 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b15e34-a207-4ea6-8220-d56caba48c72 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:32.730304 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b15e34-a207-4ea6-8220-d56caba48c72 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:32.730304 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b15e34-a207-4ea6-8220-d56caba48c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:32.871799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-43b15e34-a207-4ea6-8220-d56caba48c72 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:32.877644 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-43b15e34-a207-4ea6-8220-d56caba48c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:32.893800 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:33.041992 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 825/1656] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:32:33.152663 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc3cc26-89eb-4672-bf55-22973a9f1825 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:33.155969 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4fc3cc26-89eb-4672-bf55-22973a9f1825 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:33.311305 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:33.350367 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc3cc26-89eb-4672-bf55-22973a9f1825 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:33.357675 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c35c5f-0217-467c-b3a3-13367890f22f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:33.358595 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c35c5f-0217-467c-b3a3-13367890f22f service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:33.358926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c35c5f-0217-467c-b3a3-13367890f22f service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:33.359208 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c35c5f-0217-467c-b3a3-13367890f22f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:33.382477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c35c5f-0217-467c-b3a3-13367890f22f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:33.385360 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c35c5f-0217-467c-b3a3-13367890f22f service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:33.421900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 832/1657] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:33.558243 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b15e34-a207-4ea6-8220-d56caba48c72 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:33.559751 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-43b15e34-a207-4ea6-8220-d56caba48c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:33.583857 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b15e34-a207-4ea6-8220-d56caba48c72 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:33.603253 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e68319-9ddc-4dc0-bc5e-84165f03ad41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:33.604192 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e68319-9ddc-4dc0-bc5e-84165f03ad41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:33.604627 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e68319-9ddc-4dc0-bc5e-84165f03ad41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:33.605054 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e68319-9ddc-4dc0-bc5e-84165f03ad41 admin admin] PATH_INFO: `/v3/projects/5dc2dc3eab8d4594a4566a4220870b8f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:33.619052 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e68319-9ddc-4dc0-bc5e-84165f03ad41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dc2dc3eab8d4594a4566a4220870b8f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:33.619052 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e68319-9ddc-4dc0-bc5e-84165f03ad41 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:33.887607 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-25e68319-9ddc-4dc0-bc5e-84165f03ad41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dc2dc3eab8d4594a4566a4220870b8f', 'request_id': 'req-25e68319-9ddc-4dc0-bc5e-84165f03ad41'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:33.908297 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-25e68319-9ddc-4dc0-bc5e-84165f03ad41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dc2dc3eab8d4594a4566a4220870b8f', 'request_id': 'req-25e68319-9ddc-4dc0-bc5e-84165f03ad41'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:33.928776 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-25e68319-9ddc-4dc0-bc5e-84165f03ad41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dc2dc3eab8d4594a4566a4220870b8f', 'request_id': 'req-25e68319-9ddc-4dc0-bc5e-84165f03ad41'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:34.063315 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-25e68319-9ddc-4dc0-bc5e-84165f03ad41 admin admin] The token cache is being invalidate because project 5dc2dc3eab8d4594a4566a4220870b8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:32:34.064948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-25e68319-9ddc-4dc0-bc5e-84165f03ad41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:34.065431 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-25e68319-9ddc-4dc0-bc5e-84165f03ad41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:34.075780 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 826/1658] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:32:33 2022] DELETE /identity/v3/projects/5dc2dc3eab8d4594a4566a4220870b8f => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:32:34.080004 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c35c5f-0217-467c-b3a3-13367890f22f service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:34.098545 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c35c5f-0217-467c-b3a3-13367890f22f service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:34.176152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e68319-9ddc-4dc0-bc5e-84165f03ad41 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:34.178822 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-25e68319-9ddc-4dc0-bc5e-84165f03ad41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:34.225640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03d8e3-bbe8-44ef-92ba-33b0d81a81b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:34.226830 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03d8e3-bbe8-44ef-92ba-33b0d81a81b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:34.227269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03d8e3-bbe8-44ef-92ba-33b0d81a81b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:34.227691 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03d8e3-bbe8-44ef-92ba-33b0d81a81b7 admin admin] PATH_INFO: `/v3/users/6d74dfccb6fb47bd8265f6b7dceb25b0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:34.229585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc03d8e3-bbe8-44ef-92ba-33b0d81a81b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d74dfccb6fb47bd8265f6b7dceb25b0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:34.231815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc03d8e3-bbe8-44ef-92ba-33b0d81a81b7 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:34.329639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e68319-9ddc-4dc0-bc5e-84165f03ad41 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:34.335622 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a656813-be16-43e0-9737-4336406d8953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:34.336292 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a656813-be16-43e0-9737-4336406d8953 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:34.336576 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a656813-be16-43e0-9737-4336406d8953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:34.336852 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a656813-be16-43e0-9737-4336406d8953 admin admin] PATH_INFO: `/v3/users/9637e0dfaee84bbc8700a9951f3ba756` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:34.338211 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a656813-be16-43e0-9737-4336406d8953 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9637e0dfaee84bbc8700a9951f3ba756)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:34.340419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a656813-be16-43e0-9737-4336406d8953 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:34.486641 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cc03d8e3-bbe8-44ef-92ba-33b0d81a81b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d74dfccb6fb47bd8265f6b7dceb25b0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:34.496143 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cc03d8e3-bbe8-44ef-92ba-33b0d81a81b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d74dfccb6fb47bd8265f6b7dceb25b0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:34.582193 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cc03d8e3-bbe8-44ef-92ba-33b0d81a81b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d74dfccb6fb47bd8265f6b7dceb25b0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:34.583001 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cc03d8e3-bbe8-44ef-92ba-33b0d81a81b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d74dfccb6fb47bd8265f6b7dceb25b0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:34.607689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7a656813-be16-43e0-9737-4336406d8953 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9637e0dfaee84bbc8700a9951f3ba756'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:34.608972 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cc03d8e3-bbe8-44ef-92ba-33b0d81a81b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d74dfccb6fb47bd8265f6b7dceb25b0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:34.619833 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 833/1659] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:32:34 2022] DELETE /identity/v3/users/6d74dfccb6fb47bd8265f6b7dceb25b0 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:34.621039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03d8e3-bbe8-44ef-92ba-33b0d81a81b7 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:34.627811 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cc03d8e3-bbe8-44ef-92ba-33b0d81a81b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:34.731144 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7a656813-be16-43e0-9737-4336406d8953 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9637e0dfaee84bbc8700a9951f3ba756'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:34.731794 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7a656813-be16-43e0-9737-4336406d8953 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9637e0dfaee84bbc8700a9951f3ba756'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:34.734878 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7a656813-be16-43e0-9737-4336406d8953 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9637e0dfaee84bbc8700a9951f3ba756'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:34.761346 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-7a656813-be16-43e0-9737-4336406d8953 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9637e0dfaee84bbc8700a9951f3ba756'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:34.764672 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03d8e3-bbe8-44ef-92ba-33b0d81a81b7 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:34.773437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd6eeeb-36d0-4a2b-8126-cfec88549067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:34.774101 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6eeeb-36d0-4a2b-8126-cfec88549067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:34.774397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6eeeb-36d0-4a2b-8126-cfec88549067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:34.774666 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6eeeb-36d0-4a2b-8126-cfec88549067 admin admin] PATH_INFO: `/v3/projects/0f0b7885fff048ca80407f436430e768` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:34.775886 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd6eeeb-36d0-4a2b-8126-cfec88549067 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f0b7885fff048ca80407f436430e768)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:34.777111 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: WARNING py.warnings [None req-9fd6eeeb-36d0-4a2b-8126-cfec88549067 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 02 09:32:34.777111 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: warnings.warn(msg) Sep 02 09:32:34.777111 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: Sep 02 09:32:34.778339 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd6eeeb-36d0-4a2b-8126-cfec88549067 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:34.804940 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 827/1660] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:32:34 2022] DELETE /identity/v3/users/9637e0dfaee84bbc8700a9951f3ba756 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:32:34.807765 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a31096-03d1-40bc-8b6b-7d2392adf908 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:34.808054 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a31096-03d1-40bc-8b6b-7d2392adf908 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:34.808332 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a31096-03d1-40bc-8b6b-7d2392adf908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:34.850490 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-47a31096-03d1-40bc-8b6b-7d2392adf908 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:34.853654 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-47a31096-03d1-40bc-8b6b-7d2392adf908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:34.951701 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 828/1661] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:34.972604 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a31096-03d1-40bc-8b6b-7d2392adf908 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:34.976171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-47a31096-03d1-40bc-8b6b-7d2392adf908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:35.001481 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9fd6eeeb-36d0-4a2b-8126-cfec88549067 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f0b7885fff048ca80407f436430e768', 'request_id': 'req-9fd6eeeb-36d0-4a2b-8126-cfec88549067'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:35.070041 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a31096-03d1-40bc-8b6b-7d2392adf908 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:35.082837 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9fd6eeeb-36d0-4a2b-8126-cfec88549067 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f0b7885fff048ca80407f436430e768', 'request_id': 'req-9fd6eeeb-36d0-4a2b-8126-cfec88549067'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:35.093454 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9fd6eeeb-36d0-4a2b-8126-cfec88549067 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f0b7885fff048ca80407f436430e768', 'request_id': 'req-9fd6eeeb-36d0-4a2b-8126-cfec88549067'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:35.127280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9fd6eeeb-36d0-4a2b-8126-cfec88549067 admin admin] The token cache is being invalidate because project 0f0b7885fff048ca80407f436430e768 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:32:35.127637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9fd6eeeb-36d0-4a2b-8126-cfec88549067 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:35.128845 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9fd6eeeb-36d0-4a2b-8126-cfec88549067 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:35.137127 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 834/1662] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:32:34 2022] DELETE /identity/v3/projects/0f0b7885fff048ca80407f436430e768 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:35.142048 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd6eeeb-36d0-4a2b-8126-cfec88549067 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:35.147344 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9fd6eeeb-36d0-4a2b-8126-cfec88549067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:35.151644 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1715ac3c-cd97-4e22-83f3-34ae0c9a54b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:35.152607 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715ac3c-cd97-4e22-83f3-34ae0c9a54b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:35.152990 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715ac3c-cd97-4e22-83f3-34ae0c9a54b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:35.153273 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1715ac3c-cd97-4e22-83f3-34ae0c9a54b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:35.155630 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1715ac3c-cd97-4e22-83f3-34ae0c9a54b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:35.275365 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1715ac3c-cd97-4e22-83f3-34ae0c9a54b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:35.276874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1715ac3c-cd97-4e22-83f3-34ae0c9a54b8 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:35.290851 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 829/1663] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:35.301455 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd6eeeb-36d0-4a2b-8126-cfec88549067 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:35.303499 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f5e00-cf4c-4dd1-88a3-6ecd15cfba1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:35.304417 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f5e00-cf4c-4dd1-88a3-6ecd15cfba1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:35.304981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f5e00-cf4c-4dd1-88a3-6ecd15cfba1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:35.305409 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f5e00-cf4c-4dd1-88a3-6ecd15cfba1b admin admin] PATH_INFO: `/v3/users/cc2cdf8654d747c8bed881317eda55d5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:35.306813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f5e00-cf4c-4dd1-88a3-6ecd15cfba1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc2cdf8654d747c8bed881317eda55d5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:35.309899 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f5e00-cf4c-4dd1-88a3-6ecd15cfba1b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:35.375759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40553cd-ee16-49b2-81c1-687bec2742a2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:35.382206 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40553cd-ee16-49b2-81c1-687bec2742a2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:35.382206 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40553cd-ee16-49b2-81c1-687bec2742a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:35.426414 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d40553cd-ee16-49b2-81c1-687bec2742a2 None None] MFA Rules not processed for user `f65429de28ae41578c51a6b9099fea5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:35.429357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d40553cd-ee16-49b2-81c1-687bec2742a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:35.521677 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 830/1664] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:32:35.543819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40553cd-ee16-49b2-81c1-687bec2742a2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:35.543819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d40553cd-ee16-49b2-81c1-687bec2742a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:35.587241 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-072f5e00-cf4c-4dd1-88a3-6ecd15cfba1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc2cdf8654d747c8bed881317eda55d5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:35.605831 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-072f5e00-cf4c-4dd1-88a3-6ecd15cfba1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc2cdf8654d747c8bed881317eda55d5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:35.659233 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:35.708574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-072f5e00-cf4c-4dd1-88a3-6ecd15cfba1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc2cdf8654d747c8bed881317eda55d5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:35.709354 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-072f5e00-cf4c-4dd1-88a3-6ecd15cfba1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc2cdf8654d747c8bed881317eda55d5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:35.752607 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-072f5e00-cf4c-4dd1-88a3-6ecd15cfba1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc2cdf8654d747c8bed881317eda55d5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:35.764040 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40553cd-ee16-49b2-81c1-687bec2742a2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:35.786104 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 835/1665] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:32:35 2022] DELETE /identity/v3/users/cc2cdf8654d747c8bed881317eda55d5 => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:32:35.786798 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78af7f1-b62c-4f6e-aab8-a834061a8b77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:35.789777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78af7f1-b62c-4f6e-aab8-a834061a8b77 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:35.798416 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78af7f1-b62c-4f6e-aab8-a834061a8b77 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:35.799036 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78af7f1-b62c-4f6e-aab8-a834061a8b77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:35.804891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a78af7f1-b62c-4f6e-aab8-a834061a8b77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:35.836840 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78af7f1-b62c-4f6e-aab8-a834061a8b77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:35.843402 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78af7f1-b62c-4f6e-aab8-a834061a8b77 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:35.857119 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:35.876929 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 831/1666] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:36.074149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f5e00-cf4c-4dd1-88a3-6ecd15cfba1b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:36.075858 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-072f5e00-cf4c-4dd1-88a3-6ecd15cfba1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:36.296443 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f5e00-cf4c-4dd1-88a3-6ecd15cfba1b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:36.305614 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae919aee-c308-42f6-8ff7-931cc8c4ddeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:36.306591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae919aee-c308-42f6-8ff7-931cc8c4ddeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:36.307690 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae919aee-c308-42f6-8ff7-931cc8c4ddeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:36.308134 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae919aee-c308-42f6-8ff7-931cc8c4ddeb admin admin] PATH_INFO: `/v3/projects/7b2548e96bcf4b3b9ac9ad8cbd662efc` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:36.309598 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae919aee-c308-42f6-8ff7-931cc8c4ddeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b2548e96bcf4b3b9ac9ad8cbd662efc)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:36.312020 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae919aee-c308-42f6-8ff7-931cc8c4ddeb admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:36.510897 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ae919aee-c308-42f6-8ff7-931cc8c4ddeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b2548e96bcf4b3b9ac9ad8cbd662efc', 'request_id': 'req-ae919aee-c308-42f6-8ff7-931cc8c4ddeb'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:36.577708 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ae919aee-c308-42f6-8ff7-931cc8c4ddeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b2548e96bcf4b3b9ac9ad8cbd662efc', 'request_id': 'req-ae919aee-c308-42f6-8ff7-931cc8c4ddeb'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:36.585067 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ae919aee-c308-42f6-8ff7-931cc8c4ddeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b2548e96bcf4b3b9ac9ad8cbd662efc', 'request_id': 'req-ae919aee-c308-42f6-8ff7-931cc8c4ddeb'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:36.599935 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ae919aee-c308-42f6-8ff7-931cc8c4ddeb admin admin] The token cache is being invalidate because project 7b2548e96bcf4b3b9ac9ad8cbd662efc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:32:36.600315 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ae919aee-c308-42f6-8ff7-931cc8c4ddeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:36.601237 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ae919aee-c308-42f6-8ff7-931cc8c4ddeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:36.603164 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 836/1667] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:32:36 2022] DELETE /identity/v3/projects/7b2548e96bcf4b3b9ac9ad8cbd662efc => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:36.608551 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb0771-f94d-4391-bd22-bd00c23777a2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:36.609072 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb0771-f94d-4391-bd22-bd00c23777a2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:36.609410 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb0771-f94d-4391-bd22-bd00c23777a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:36.639530 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f6fb0771-f94d-4391-bd22-bd00c23777a2 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:36.639530 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f6fb0771-f94d-4391-bd22-bd00c23777a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:36.643085 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae919aee-c308-42f6-8ff7-931cc8c4ddeb admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:36.643085 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ae919aee-c308-42f6-8ff7-931cc8c4ddeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:36.774039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae919aee-c308-42f6-8ff7-931cc8c4ddeb admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:36.780585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f208342-850f-476c-98f2-f3f954b9d1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:36.781712 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f208342-850f-476c-98f2-f3f954b9d1a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:36.782062 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f208342-850f-476c-98f2-f3f954b9d1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:36.782577 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f208342-850f-476c-98f2-f3f954b9d1a6 admin admin] PATH_INFO: `/v3/projects/991f0a2854874d87805ab613b34fc7d9` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:36.784210 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 832/1668] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:36.785330 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb0771-f94d-4391-bd22-bd00c23777a2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:36.793042 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f208342-850f-476c-98f2-f3f954b9d1a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=991f0a2854874d87805ab613b34fc7d9)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:36.793469 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f6fb0771-f94d-4391-bd22-bd00c23777a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:36.795444 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f208342-850f-476c-98f2-f3f954b9d1a6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:36.819329 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb0771-f94d-4391-bd22-bd00c23777a2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:36.826962 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f68595-67bb-4ce0-847b-2c4380294b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:36.827098 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f68595-67bb-4ce0-847b-2c4380294b48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:36.827382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f68595-67bb-4ce0-847b-2c4380294b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:36.827673 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f68595-67bb-4ce0-847b-2c4380294b48 admin admin] PATH_INFO: `/v3/projects/62d5c9f1ea8b4929860a24a3189e0379` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:36.838316 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f68595-67bb-4ce0-847b-2c4380294b48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62d5c9f1ea8b4929860a24a3189e0379)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:36.838839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f68595-67bb-4ce0-847b-2c4380294b48 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:36.930690 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-0f208342-850f-476c-98f2-f3f954b9d1a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '991f0a2854874d87805ab613b34fc7d9', 'request_id': 'req-0f208342-850f-476c-98f2-f3f954b9d1a6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:36.996359 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d8f68595-67bb-4ce0-847b-2c4380294b48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62d5c9f1ea8b4929860a24a3189e0379', 'request_id': 'req-d8f68595-67bb-4ce0-847b-2c4380294b48'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:37.005572 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d8f68595-67bb-4ce0-847b-2c4380294b48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62d5c9f1ea8b4929860a24a3189e0379', 'request_id': 'req-d8f68595-67bb-4ce0-847b-2c4380294b48'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:37.014546 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d8f68595-67bb-4ce0-847b-2c4380294b48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62d5c9f1ea8b4929860a24a3189e0379', 'request_id': 'req-d8f68595-67bb-4ce0-847b-2c4380294b48'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:37.049279 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-0f208342-850f-476c-98f2-f3f954b9d1a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '991f0a2854874d87805ab613b34fc7d9', 'request_id': 'req-0f208342-850f-476c-98f2-f3f954b9d1a6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:37.057070 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-0f208342-850f-476c-98f2-f3f954b9d1a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '991f0a2854874d87805ab613b34fc7d9', 'request_id': 'req-0f208342-850f-476c-98f2-f3f954b9d1a6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:37.072795 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-0f208342-850f-476c-98f2-f3f954b9d1a6 admin admin] The token cache is being invalidate because project 991f0a2854874d87805ab613b34fc7d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:32:37.073202 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-0f208342-850f-476c-98f2-f3f954b9d1a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:37.073929 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-0f208342-850f-476c-98f2-f3f954b9d1a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:37.075811 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 837/1669] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:32:36 2022] DELETE /identity/v3/projects/991f0a2854874d87805ab613b34fc7d9 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:37.077284 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb692700-80e0-438c-980c-5f9f99c32402 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:37.077633 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb692700-80e0-438c-980c-5f9f99c32402 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:37.077973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb692700-80e0-438c-980c-5f9f99c32402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:37.098957 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-cb692700-80e0-438c-980c-5f9f99c32402 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:37.100709 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cb692700-80e0-438c-980c-5f9f99c32402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:37.139800 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d8f68595-67bb-4ce0-847b-2c4380294b48 admin admin] The token cache is being invalidate because project 62d5c9f1ea8b4929860a24a3189e0379 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:32:37.143537 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d8f68595-67bb-4ce0-847b-2c4380294b48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:37.143537 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d8f68595-67bb-4ce0-847b-2c4380294b48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:37.145894 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 833/1670] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:32:36 2022] DELETE /identity/v3/projects/62d5c9f1ea8b4929860a24a3189e0379 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:37.148713 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbba3b1-bc60-415e-8aa0-f0660f11ba09 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:37.150536 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbba3b1-bc60-415e-8aa0-f0660f11ba09 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:37.156655 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbba3b1-bc60-415e-8aa0-f0660f11ba09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:37.204253 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ecbba3b1-bc60-415e-8aa0-f0660f11ba09 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:37.205740 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ecbba3b1-bc60-415e-8aa0-f0660f11ba09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:37.214026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:37.253078 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 838/1671] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:37.253078 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb692700-80e0-438c-980c-5f9f99c32402 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:37.255122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cb692700-80e0-438c-980c-5f9f99c32402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:37.280457 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:37.290000 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb692700-80e0-438c-980c-5f9f99c32402 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:37.293189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb0267-cd55-461e-9e3e-13947e65da5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:37.294050 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb0267-cd55-461e-9e3e-13947e65da5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:37.294436 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb0267-cd55-461e-9e3e-13947e65da5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:37.294855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb0267-cd55-461e-9e3e-13947e65da5c admin admin] PATH_INFO: `/v3/users/e548f129bdc04a3bbc2ed7618d742b08` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:37.295632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 834/1672] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:37.296524 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcb0267-cd55-461e-9e3e-13947e65da5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e548f129bdc04a3bbc2ed7618d742b08)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:37.297208 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbba3b1-bc60-415e-8aa0-f0660f11ba09 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:37.299231 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcb0267-cd55-461e-9e3e-13947e65da5c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:37.307884 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbba3b1-bc60-415e-8aa0-f0660f11ba09 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:37.313136 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7b43a1-3698-4d36-aad7-8a42d47a1a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:37.313973 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b43a1-3698-4d36-aad7-8a42d47a1a7a admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:37.314398 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b43a1-3698-4d36-aad7-8a42d47a1a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:37.314683 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7b43a1-3698-4d36-aad7-8a42d47a1a7a admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:37.315560 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b43a1-3698-4d36-aad7-8a42d47a1a7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:32:37.316011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b43a1-3698-4d36-aad7-8a42d47a1a7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:37.317879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7b43a1-3698-4d36-aad7-8a42d47a1a7a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:37.336534 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 835/1673] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:32:37.338170 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c2015-1eb1-46b4-aed6-cad9be4b8d3e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:37.338629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c2015-1eb1-46b4-aed6-cad9be4b8d3e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:37.339100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c2015-1eb1-46b4-aed6-cad9be4b8d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:37.367758 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-336c2015-1eb1-46b4-aed6-cad9be4b8d3e None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:37.369812 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-336c2015-1eb1-46b4-aed6-cad9be4b8d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:37.388696 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 836/1674] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:37.389806 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c2015-1eb1-46b4-aed6-cad9be4b8d3e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:37.403360 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c2015-1eb1-46b4-aed6-cad9be4b8d3e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:37.409085 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c37c50-d18a-4080-8922-82c8bdd2dd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:37.409900 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c37c50-d18a-4080-8922-82c8bdd2dd03 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:37.410354 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c37c50-d18a-4080-8922-82c8bdd2dd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:37.410854 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c37c50-d18a-4080-8922-82c8bdd2dd03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:37.411637 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c37c50-d18a-4080-8922-82c8bdd2dd03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:37.415357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c37c50-d18a-4080-8922-82c8bdd2dd03 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:37.539768 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 837/1675] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:32:37 2022] POST /identity/v3/projects => generated 355 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:37.543053 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c37c50-d18a-4080-8922-82c8bdd2dd03 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:37.567273 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9bcb0267-cd55-461e-9e3e-13947e65da5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e548f129bdc04a3bbc2ed7618d742b08'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:37.583277 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9bcb0267-cd55-461e-9e3e-13947e65da5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e548f129bdc04a3bbc2ed7618d742b08'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:37.672542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c37c50-d18a-4080-8922-82c8bdd2dd03 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:37.679471 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad8784-7912-443b-9a92-be6590674578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:37.680815 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad8784-7912-443b-9a92-be6590674578 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:37.682799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad8784-7912-443b-9a92-be6590674578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:37.682916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9bcb0267-cd55-461e-9e3e-13947e65da5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e548f129bdc04a3bbc2ed7618d742b08'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:37.683366 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ad8784-7912-443b-9a92-be6590674578 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:37.683595 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9bcb0267-cd55-461e-9e3e-13947e65da5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e548f129bdc04a3bbc2ed7618d742b08'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:37.684348 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad8784-7912-443b-9a92-be6590674578 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:32:37.684815 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad8784-7912-443b-9a92-be6590674578 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:37.688133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ad8784-7912-443b-9a92-be6590674578 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:37.705768 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 838/1676] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:32:37.707418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad8784-7912-443b-9a92-be6590674578 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:37.711264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b5ad8784-7912-443b-9a92-be6590674578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:37.712274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9bcb0267-cd55-461e-9e3e-13947e65da5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e548f129bdc04a3bbc2ed7618d742b08'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:37.727938 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ad8784-7912-443b-9a92-be6590674578 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:37.731558 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 839/1677] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:32:37 2022] DELETE /identity/v3/users/e548f129bdc04a3bbc2ed7618d742b08 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:37.732477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb0267-cd55-461e-9e3e-13947e65da5c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:37.733517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3d00e-ec04-4c61-ad9d-43fb355f805d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:37.734640 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3d00e-ec04-4c61-ad9d-43fb355f805d admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:37.734898 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9bcb0267-cd55-461e-9e3e-13947e65da5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:37.735313 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3d00e-ec04-4c61-ad9d-43fb355f805d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:37.735744 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3d00e-ec04-4c61-ad9d-43fb355f805d admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:37.736664 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3d00e-ec04-4c61-ad9d-43fb355f805d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:37.739225 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3d00e-ec04-4c61-ad9d-43fb355f805d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:37.922131 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 839/1678] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:32:37 2022] POST /identity/v3/users => generated 278 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:37.927997 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3d00e-ec04-4c61-ad9d-43fb355f805d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:37.969306 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3d00e-ec04-4c61-ad9d-43fb355f805d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:37.995374 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f91f4-55d2-4c52-8fc7-00af4cba3203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:37.996051 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f91f4-55d2-4c52-8fc7-00af4cba3203 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:37.996329 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f91f4-55d2-4c52-8fc7-00af4cba3203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:37.996814 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f91f4-55d2-4c52-8fc7-00af4cba3203 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:37.997493 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f91f4-55d2-4c52-8fc7-00af4cba3203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:37.999493 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f91f4-55d2-4c52-8fc7-00af4cba3203 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:38.003868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb0267-cd55-461e-9e3e-13947e65da5c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:38.021267 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d24a5-e676-413a-bc0b-a43c32bcefdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:38.024106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d24a5-e676-413a-bc0b-a43c32bcefdf admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:38.024600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d24a5-e676-413a-bc0b-a43c32bcefdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:38.025061 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d24a5-e676-413a-bc0b-a43c32bcefdf admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:38.026169 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d24a5-e676-413a-bc0b-a43c32bcefdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:38.034244 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d24a5-e676-413a-bc0b-a43c32bcefdf admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:38.050006 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 840/1679] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:37 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:38.054042 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f91f4-55d2-4c52-8fc7-00af4cba3203 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:38.081465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f91f4-55d2-4c52-8fc7-00af4cba3203 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:38.081465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f200c-bbe7-4e36-aebd-5a57e13ab6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:38.081465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f200c-bbe7-4e36-aebd-5a57e13ab6c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:38.081465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f200c-bbe7-4e36-aebd-5a57e13ab6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:38.081769 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f200c-bbe7-4e36-aebd-5a57e13ab6c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:38.081951 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f200c-bbe7-4e36-aebd-5a57e13ab6c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:38.084709 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f200c-bbe7-4e36-aebd-5a57e13ab6c0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:38.097186 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 841/1680] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:38.101473 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f200c-bbe7-4e36-aebd-5a57e13ab6c0 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:38.121270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f200c-bbe7-4e36-aebd-5a57e13ab6c0 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:38.127164 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d636b-deca-4cec-bc3c-d9fc9fb22acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:38.128033 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d636b-deca-4cec-bc3c-d9fc9fb22acf admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:38.128427 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d636b-deca-4cec-bc3c-d9fc9fb22acf admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:38.128810 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d636b-deca-4cec-bc3c-d9fc9fb22acf admin admin] PATH_INFO: `/v3/projects/f8dea0a90aaf402da76b76f721da3878/users/822d68bc5c044db3a0842bde2fdf1a73/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:38.153938 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195d636b-deca-4cec-bc3c-d9fc9fb22acf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8dea0a90aaf402da76b76f721da3878, user_id=822d68bc5c044db3a0842bde2fdf1a73, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:38.157221 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195d636b-deca-4cec-bc3c-d9fc9fb22acf admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:38.171370 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 840/1681] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:32:37 2022] POST /identity/v3/projects => generated 367 bytes in 439 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:38.175603 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d24a5-e676-413a-bc0b-a43c32bcefdf admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:38.298167 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d24a5-e676-413a-bc0b-a43c32bcefdf admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:38.306307 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 842/1682] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:38 2022] PUT /identity/v3/projects/f8dea0a90aaf402da76b76f721da3878/users/822d68bc5c044db3a0842bde2fdf1a73/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:38.325240 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345db9a-bfb1-4ee5-93b2-25c1d1e49ead None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:38.325794 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345db9a-bfb1-4ee5-93b2-25c1d1e49ead None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:38.326072 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345db9a-bfb1-4ee5-93b2-25c1d1e49ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:38.388830 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7345db9a-bfb1-4ee5-93b2-25c1d1e49ead None None] MFA Rules not processed for user `822d68bc5c044db3a0842bde2fdf1a73`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:38.390302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7345db9a-bfb1-4ee5-93b2-25c1d1e49ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:38.455858 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e7ff4-0b93-480b-b2ec-316f4f0c0ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:38.456490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e7ff4-0b93-480b-b2ec-316f4f0c0ae1 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:38.456768 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e7ff4-0b93-480b-b2ec-316f4f0c0ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:38.457035 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e7ff4-0b93-480b-b2ec-316f4f0c0ae1 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:38.457675 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e7ff4-0b93-480b-b2ec-316f4f0c0ae1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:38.461236 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e7ff4-0b93-480b-b2ec-316f4f0c0ae1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:38.562585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 841/1683] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:32:38 2022] POST /identity/v3/users => generated 284 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:32:38.562585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e7ff4-0b93-480b-b2ec-316f4f0c0ae1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:38.574975 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 843/1684] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:38 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:32:38.579205 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7345db9a-bfb1-4ee5-93b2-25c1d1e49ead None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:38.585958 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e7ff4-0b93-480b-b2ec-316f4f0c0ae1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:38.597800 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab205e0-7619-4f4f-844d-aca387e21b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:38.599732 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab205e0-7619-4f4f-844d-aca387e21b4f admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:38.600108 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab205e0-7619-4f4f-844d-aca387e21b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:38.600486 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab205e0-7619-4f4f-844d-aca387e21b4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:38.601835 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab205e0-7619-4f4f-844d-aca387e21b4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:38.606967 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab205e0-7619-4f4f-844d-aca387e21b4f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:38.608984 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7345db9a-bfb1-4ee5-93b2-25c1d1e49ead None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:38.622001 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88105964-999c-4456-8f37-f1bebe9ac2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:38.622938 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88105964-999c-4456-8f37-f1bebe9ac2c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:38.623354 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88105964-999c-4456-8f37-f1bebe9ac2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:38.623772 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88105964-999c-4456-8f37-f1bebe9ac2c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:38.624571 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88105964-999c-4456-8f37-f1bebe9ac2c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:38.626748 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88105964-999c-4456-8f37-f1bebe9ac2c5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:38.637101 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 842/1685] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:38.640983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab205e0-7619-4f4f-844d-aca387e21b4f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:38.660160 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab205e0-7619-4f4f-844d-aca387e21b4f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:38.673341 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b00c81-589d-4d02-ab67-8e52987d2c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:38.674081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b00c81-589d-4d02-ab67-8e52987d2c4a admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:38.674469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b00c81-589d-4d02-ab67-8e52987d2c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:38.674911 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b00c81-589d-4d02-ab67-8e52987d2c4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:38.675912 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b00c81-589d-4d02-ab67-8e52987d2c4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:38.682742 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b00c81-589d-4d02-ab67-8e52987d2c4a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:38.717978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 843/1686] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:32:38.722832 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b00c81-589d-4d02-ab67-8e52987d2c4a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:38.748494 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 844/1687] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:32:38 2022] POST /identity/v3/projects => generated 355 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:32:38.752579 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a22c53-7b28-4b0f-816c-9ce83a065e3b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:38.752959 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a22c53-7b28-4b0f-816c-9ce83a065e3b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:38.753158 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a22c53-7b28-4b0f-816c-9ce83a065e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:38.765926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b00c81-589d-4d02-ab67-8e52987d2c4a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:38.825186 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d5a22c53-7b28-4b0f-816c-9ce83a065e3b None None] MFA Rules not processed for user `c79aaf1f72634eefb5e89c8005afe54b`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:38.830342 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d5a22c53-7b28-4b0f-816c-9ce83a065e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:38.893030 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a797e9d4-e43e-492c-8e0b-2538c7d00379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:38.894190 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797e9d4-e43e-492c-8e0b-2538c7d00379 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:38.894801 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797e9d4-e43e-492c-8e0b-2538c7d00379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:38.895415 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797e9d4-e43e-492c-8e0b-2538c7d00379 admin admin] PATH_INFO: `/v3/projects/9c0acff1856e47cab0a33d77a65899eb/users/5f5b2ef048414faabeb7d6df013a5964/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:38.917302 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a797e9d4-e43e-492c-8e0b-2538c7d00379 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c0acff1856e47cab0a33d77a65899eb, user_id=5f5b2ef048414faabeb7d6df013a5964, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:38.921474 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a797e9d4-e43e-492c-8e0b-2538c7d00379 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:38.978097 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 845/1688] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:38 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:38.978097 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a22c53-7b28-4b0f-816c-9ce83a065e3b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:38.991979 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a22c53-7b28-4b0f-816c-9ce83a065e3b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:39.012616 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a2cb7-1cc4-4d6c-8ac7-427a48206ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:39.013344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a2cb7-1cc4-4d6c-8ac7-427a48206ee8 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:39.015512 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a2cb7-1cc4-4d6c-8ac7-427a48206ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:39.019565 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a2cb7-1cc4-4d6c-8ac7-427a48206ee8 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:39.019565 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676a2cb7-1cc4-4d6c-8ac7-427a48206ee8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:39.024346 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676a2cb7-1cc4-4d6c-8ac7-427a48206ee8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:39.078266 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 844/1689] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:38 2022] PUT /identity/v3/projects/9c0acff1856e47cab0a33d77a65899eb/users/5f5b2ef048414faabeb7d6df013a5964/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:39.078266 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1447e10-8377-4e0c-8a12-3b194005eeda None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:39.078266 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1447e10-8377-4e0c-8a12-3b194005eeda None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:39.078266 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1447e10-8377-4e0c-8a12-3b194005eeda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:39.127378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b1447e10-8377-4e0c-8a12-3b194005eeda None None] MFA Rules not processed for user `c79aaf1f72634eefb5e89c8005afe54b`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:39.129491 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b1447e10-8377-4e0c-8a12-3b194005eeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:39.133931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 846/1690] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:32:38 2022] POST /identity/v3/users => generated 284 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:39.135639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e611e-d560-4926-bda3-f9560c72d7f7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:39.136207 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e611e-d560-4926-bda3-f9560c72d7f7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:39.137308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e611e-d560-4926-bda3-f9560c72d7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:39.193152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d25e611e-d560-4926-bda3-f9560c72d7f7 None None] MFA Rules not processed for user `5f5b2ef048414faabeb7d6df013a5964`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:39.197188 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d25e611e-d560-4926-bda3-f9560c72d7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:39.273970 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 845/1691] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:39.283811 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1447e10-8377-4e0c-8a12-3b194005eeda None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:39.359155 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:39.365116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 847/1692] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:39.366559 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e611e-d560-4926-bda3-f9560c72d7f7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:39.368054 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d25e611e-d560-4926-bda3-f9560c72d7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:39.477013 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:39.513607 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e611e-d560-4926-bda3-f9560c72d7f7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:39.525335 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818740c1-aedc-42d6-ae25-e1e293b5c9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:39.526486 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818740c1-aedc-42d6-ae25-e1e293b5c9e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:39.526740 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818740c1-aedc-42d6-ae25-e1e293b5c9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:39.530851 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818740c1-aedc-42d6-ae25-e1e293b5c9e4 admin admin] PATH_INFO: `/v3/projects/e2650310037e4858852a02b8cf19e7bd` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:39.530851 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818740c1-aedc-42d6-ae25-e1e293b5c9e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2650310037e4858852a02b8cf19e7bd)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:39.533465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818740c1-aedc-42d6-ae25-e1e293b5c9e4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:39.569290 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1447e10-8377-4e0c-8a12-3b194005eeda None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:39.589786 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2b1e5-bbf8-4f1e-993a-b451c515c840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:39.590604 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2b1e5-bbf8-4f1e-993a-b451c515c840 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:39.590924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2b1e5-bbf8-4f1e-993a-b451c515c840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:39.591199 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2b1e5-bbf8-4f1e-993a-b451c515c840 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:39.594189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2b1e5-bbf8-4f1e-993a-b451c515c840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:39.598682 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2b1e5-bbf8-4f1e-993a-b451c515c840 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:39.619019 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 846/1693] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:39 2022] GET /identity/v3/roles => generated 1420 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:39.624271 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2b1e5-bbf8-4f1e-993a-b451c515c840 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:39.628428 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d9b2b1e5-bbf8-4f1e-993a-b451c515c840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:39.715830 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-818740c1-aedc-42d6-ae25-e1e293b5c9e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2650310037e4858852a02b8cf19e7bd', 'request_id': 'req-818740c1-aedc-42d6-ae25-e1e293b5c9e4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:39.732234 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-818740c1-aedc-42d6-ae25-e1e293b5c9e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2650310037e4858852a02b8cf19e7bd', 'request_id': 'req-818740c1-aedc-42d6-ae25-e1e293b5c9e4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:39.758849 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-818740c1-aedc-42d6-ae25-e1e293b5c9e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2650310037e4858852a02b8cf19e7bd', 'request_id': 'req-818740c1-aedc-42d6-ae25-e1e293b5c9e4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:39.794755 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:39.828674 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2b1e5-bbf8-4f1e-993a-b451c515c840 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:39.869990 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-818740c1-aedc-42d6-ae25-e1e293b5c9e4 admin admin] The token cache is being invalidate because project e2650310037e4858852a02b8cf19e7bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:32:39.878769 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-818740c1-aedc-42d6-ae25-e1e293b5c9e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:39.878769 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-818740c1-aedc-42d6-ae25-e1e293b5c9e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:39.878769 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 848/1694] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:32:39 2022] DELETE /identity/v3/projects/e2650310037e4858852a02b8cf19e7bd => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:39.878769 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818740c1-aedc-42d6-ae25-e1e293b5c9e4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:39.885194 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-818740c1-aedc-42d6-ae25-e1e293b5c9e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:40.009400 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419963a-b6a3-48a1-bd61-31351569db72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:40.010647 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419963a-b6a3-48a1-bd61-31351569db72 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:40.010647 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419963a-b6a3-48a1-bd61-31351569db72 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:40.010647 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419963a-b6a3-48a1-bd61-31351569db72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:40.014354 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c419963a-b6a3-48a1-bd61-31351569db72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:40.083720 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818740c1-aedc-42d6-ae25-e1e293b5c9e4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:40.100435 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f21dd-fa5d-4150-aa9e-a1a52eb5a396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:40.101080 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f21dd-fa5d-4150-aa9e-a1a52eb5a396 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:40.101358 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f21dd-fa5d-4150-aa9e-a1a52eb5a396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:40.101637 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f21dd-fa5d-4150-aa9e-a1a52eb5a396 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:40.102282 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f21dd-fa5d-4150-aa9e-a1a52eb5a396 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:40.105300 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f21dd-fa5d-4150-aa9e-a1a52eb5a396 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:40.190880 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c419963a-b6a3-48a1-bd61-31351569db72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:40.192732 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c419963a-b6a3-48a1-bd61-31351569db72 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:40.217015 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 847/1695] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:39 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:40.218110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419963a-b6a3-48a1-bd61-31351569db72 service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:40.221988 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c419963a-b6a3-48a1-bd61-31351569db72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:40.245271 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419963a-b6a3-48a1-bd61-31351569db72 service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:40.249764 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363843c-35cc-42a8-8751-7d4d5d2f7661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:40.250478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363843c-35cc-42a8-8751-7d4d5d2f7661 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:40.250863 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363843c-35cc-42a8-8751-7d4d5d2f7661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:40.251149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363843c-35cc-42a8-8751-7d4d5d2f7661 admin admin] PATH_INFO: `/v3/projects/ba3f6ec8d61849808f6990dcb92aed27/users/9fff8e743d484bc686717cfbe716329f/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:40.268744 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363843c-35cc-42a8-8751-7d4d5d2f7661 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba3f6ec8d61849808f6990dcb92aed27, user_id=9fff8e743d484bc686717cfbe716329f, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:40.271514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363843c-35cc-42a8-8751-7d4d5d2f7661 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:40.295794 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 849/1696] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:32:39 2022] POST /identity/v3/projects => generated 369 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:40.297836 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9953c-c64c-4374-88c9-fbe57ac4bba2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:40.298424 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9953c-c64c-4374-88c9-fbe57ac4bba2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:40.299319 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be9953c-c64c-4374-88c9-fbe57ac4bba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:40.324545 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8be9953c-c64c-4374-88c9-fbe57ac4bba2 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:40.326096 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8be9953c-c64c-4374-88c9-fbe57ac4bba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:40.367631 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 848/1697] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:40 2022] PUT /identity/v3/projects/ba3f6ec8d61849808f6990dcb92aed27/users/9fff8e743d484bc686717cfbe716329f/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:40.371215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b09aff-9f18-497e-baa1-3b014707e472 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:40.373717 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b09aff-9f18-497e-baa1-3b014707e472 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:40.374042 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b09aff-9f18-497e-baa1-3b014707e472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:40.485596 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-07b09aff-9f18-497e-baa1-3b014707e472 None None] MFA Rules not processed for user `d4d4f092761d44a2a30f0e1d044888b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:40.487693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-07b09aff-9f18-497e-baa1-3b014707e472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:40.520898 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 850/1698] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:40.524341 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9953c-c64c-4374-88c9-fbe57ac4bba2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:40.556063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be9953c-c64c-4374-88c9-fbe57ac4bba2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:40.654668 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 849/1699] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:40.655718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b09aff-9f18-497e-baa1-3b014707e472 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:40.682650 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9914ef7e-f8e6-4d21-992c-7e710e11d70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:40.683677 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914ef7e-f8e6-4d21-992c-7e710e11d70c admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:40.689173 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914ef7e-f8e6-4d21-992c-7e710e11d70c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:40.689720 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914ef7e-f8e6-4d21-992c-7e710e11d70c admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:40.693078 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9914ef7e-f8e6-4d21-992c-7e710e11d70c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:40.696792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9914ef7e-f8e6-4d21-992c-7e710e11d70c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:40.719900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:40.783569 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 851/1700] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:32:40 2022] POST /identity/v3/users => generated 291 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:40.788283 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c493f5e-bd2a-40b1-b4a0-54f583d33d2f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:40.788598 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c493f5e-bd2a-40b1-b4a0-54f583d33d2f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:40.788863 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c493f5e-bd2a-40b1-b4a0-54f583d33d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:40.813086 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b09aff-9f18-497e-baa1-3b014707e472 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:40.821871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ade504-68c8-4956-a01d-fdd61af86dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:40.828737 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ade504-68c8-4956-a01d-fdd61af86dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:40.828737 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ade504-68c8-4956-a01d-fdd61af86dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:40.828737 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ade504-68c8-4956-a01d-fdd61af86dc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:40.828737 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ade504-68c8-4956-a01d-fdd61af86dc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:40.828737 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ade504-68c8-4956-a01d-fdd61af86dc3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:40.829096 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4c493f5e-bd2a-40b1-b4a0-54f583d33d2f None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:40.829096 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4c493f5e-bd2a-40b1-b4a0-54f583d33d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:40.843626 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 850/1701] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:40.845439 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1b36c-6300-428e-a80d-b393aae7e640 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:40.846171 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1b36c-6300-428e-a80d-b393aae7e640 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:40.846945 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1b36c-6300-428e-a80d-b393aae7e640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:40.852462 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 852/1702] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:40.857490 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c493f5e-bd2a-40b1-b4a0-54f583d33d2f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:40.875014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c493f5e-bd2a-40b1-b4a0-54f583d33d2f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:40.886736 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d588ba9-f942-4cc4-ade0-286abab7a92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:40.887556 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d588ba9-f942-4cc4-ade0-286abab7a92d admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:40.888292 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-11e1b36c-6300-428e-a80d-b393aae7e640 None None] MFA Rules not processed for user `d4d4f092761d44a2a30f0e1d044888b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:40.888679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d588ba9-f942-4cc4-ade0-286abab7a92d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:40.889391 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d588ba9-f942-4cc4-ade0-286abab7a92d admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:40.890267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d588ba9-f942-4cc4-ade0-286abab7a92d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:40.893918 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-11e1b36c-6300-428e-a80d-b393aae7e640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:40.897959 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d588ba9-f942-4cc4-ade0-286abab7a92d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:40.912799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 851/1703] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:40.916657 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 853/1704] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:40 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:40.917651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d588ba9-f942-4cc4-ade0-286abab7a92d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:40.921613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1b36c-6300-428e-a80d-b393aae7e640 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:40.940973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1b36c-6300-428e-a80d-b393aae7e640 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:40.942602 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d588ba9-f942-4cc4-ade0-286abab7a92d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:40.946204 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669e923-208a-4676-a381-7a18a7038f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:40.947008 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669e923-208a-4676-a381-7a18a7038f99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:40.947339 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669e923-208a-4676-a381-7a18a7038f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:40.947622 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669e923-208a-4676-a381-7a18a7038f99 admin admin] PATH_INFO: `/v3/projects/ba3f6ec8d61849808f6990dcb92aed27/users/9fff8e743d484bc686717cfbe716329f/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:40.949732 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6669e923-208a-4676-a381-7a18a7038f99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba3f6ec8d61849808f6990dcb92aed27, user_id=9fff8e743d484bc686717cfbe716329f, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:40.953193 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6669e923-208a-4676-a381-7a18a7038f99 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:40.960599 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c4e2ed-a6dc-4d86-aa88-86555a67ff62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:40.961676 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4e2ed-a6dc-4d86-aa88-86555a67ff62 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:40.962129 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4e2ed-a6dc-4d86-aa88-86555a67ff62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:40.962561 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4e2ed-a6dc-4d86-aa88-86555a67ff62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:40.963607 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c4e2ed-a6dc-4d86-aa88-86555a67ff62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:32:40.964074 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c4e2ed-a6dc-4d86-aa88-86555a67ff62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:40.965950 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c4e2ed-a6dc-4d86-aa88-86555a67ff62 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:40.970993 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 852/1705] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:40 2022] PUT /identity/v3/projects/ba3f6ec8d61849808f6990dcb92aed27/users/9fff8e743d484bc686717cfbe716329f/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:40.978897 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e6495-c9a7-470f-840a-93a0602bccda None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:40.978897 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e6495-c9a7-470f-840a-93a0602bccda None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:40.978897 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e6495-c9a7-470f-840a-93a0602bccda None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:40.978897 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 853/1706] 10.222.0.57 () {54 vars in 912 bytes} [Fri Sep 2 09:32:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:32:40.978897 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e6495-c9a7-470f-840a-93a0602bccda None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:41.014904 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 854/1707] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:32:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:32:41.015954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c4e2ed-a6dc-4d86-aa88-86555a67ff62 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:41.022186 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c9c4e2ed-a6dc-4d86-aa88-86555a67ff62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:41.112659 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e6495-c9a7-470f-840a-93a0602bccda None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:41.119552 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cbca07-a434-47db-b4db-3c949e119274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:41.119552 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbca07-a434-47db-b4db-3c949e119274 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:41.119552 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbca07-a434-47db-b4db-3c949e119274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:41.119552 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbca07-a434-47db-b4db-3c949e119274 admin admin] PATH_INFO: `/v3/projects/c7674da09afa4458bd89cc0fc57d448d/users/5a931551170042ee90bb9eb33a1a2838/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:41.135531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cbca07-a434-47db-b4db-3c949e119274 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7674da09afa4458bd89cc0fc57d448d, user_id=5a931551170042ee90bb9eb33a1a2838, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:41.138346 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cbca07-a434-47db-b4db-3c949e119274 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:41.172602 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c4e2ed-a6dc-4d86-aa88-86555a67ff62 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:41.242369 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 854/1708] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:40 2022] PUT /identity/v3/projects/c7674da09afa4458bd89cc0fc57d448d/users/5a931551170042ee90bb9eb33a1a2838/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:41.243739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963567c1-902c-4db7-9cb8-7d79c4ca0044 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:41.243897 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963567c1-902c-4db7-9cb8-7d79c4ca0044 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:41.244311 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963567c1-902c-4db7-9cb8-7d79c4ca0044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:41.260122 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d8d25-3a2b-4464-a403-5f79a3bb3bc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:41.261030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d8d25-3a2b-4464-a403-5f79a3bb3bc4 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:41.261448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d8d25-3a2b-4464-a403-5f79a3bb3bc4 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:41.261733 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d8d25-3a2b-4464-a403-5f79a3bb3bc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:41.291983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-963567c1-902c-4db7-9cb8-7d79c4ca0044 None None] MFA Rules not processed for user `dfea2ee0c9e84b48ac9bafe3173d937a`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:41.293603 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-963567c1-902c-4db7-9cb8-7d79c4ca0044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:41.300203 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:41.336107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d8d25-3a2b-4464-a403-5f79a3bb3bc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:41.338379 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d8d25-3a2b-4464-a403-5f79a3bb3bc4 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:41.352632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 855/1709] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:41.357587 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 855/1710] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Sep 2 09:32:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:41.367983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963567c1-902c-4db7-9cb8-7d79c4ca0044 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:41.368998 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c34cb-63a0-4448-b17b-7d25b2a656a3 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:41.369844 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c34cb-63a0-4448-b17b-7d25b2a656a3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:41.370532 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c34cb-63a0-4448-b17b-7d25b2a656a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:41.408085 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9e1c34cb-63a0-4448-b17b-7d25b2a656a3 None None] MFA Rules not processed for user `9fff8e743d484bc686717cfbe716329f`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:41.409452 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9e1c34cb-63a0-4448-b17b-7d25b2a656a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:41.463385 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963567c1-902c-4db7-9cb8-7d79c4ca0044 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:41.469655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7f099-6c39-40a7-865b-29e7fdce5f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:41.470457 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7f099-6c39-40a7-865b-29e7fdce5f77 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:41.470757 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7f099-6c39-40a7-865b-29e7fdce5f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:41.471037 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7f099-6c39-40a7-865b-29e7fdce5f77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:41.472149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed7f099-6c39-40a7-865b-29e7fdce5f77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:41.474363 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed7f099-6c39-40a7-865b-29e7fdce5f77 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:41.536245 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 856/1711] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:41 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:41.537765 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11636885-6644-451b-a3b2-f1366c0e09a1 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:41.538058 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11636885-6644-451b-a3b2-f1366c0e09a1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:41.538353 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11636885-6644-451b-a3b2-f1366c0e09a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:41.562737 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-11636885-6644-451b-a3b2-f1366c0e09a1 None None] MFA Rules not processed for user `f65429de28ae41578c51a6b9099fea5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:41.570961 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-11636885-6644-451b-a3b2-f1366c0e09a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:41.573306 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 856/1712] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:32:41 2022] POST /identity/v3/projects => generated 389 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:41.577352 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7f099-6c39-40a7-865b-29e7fdce5f77 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:41.640995 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 857/1713] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:41.646925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11636885-6644-451b-a3b2-f1366c0e09a1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:41.646925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-11636885-6644-451b-a3b2-f1366c0e09a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:41.658698 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7f099-6c39-40a7-865b-29e7fdce5f77 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:41.665867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa48e88-c19b-4c0d-a2e9-57d21b2840af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:41.666504 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa48e88-c19b-4c0d-a2e9-57d21b2840af admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:41.666822 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa48e88-c19b-4c0d-a2e9-57d21b2840af admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:41.667147 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa48e88-c19b-4c0d-a2e9-57d21b2840af admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:41.668572 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa48e88-c19b-4c0d-a2e9-57d21b2840af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:41.670240 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa48e88-c19b-4c0d-a2e9-57d21b2840af admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:41.689896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 857/1714] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:41.693882 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa48e88-c19b-4c0d-a2e9-57d21b2840af admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:41.693882 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-eaa48e88-c19b-4c0d-a2e9-57d21b2840af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:41.729217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11636885-6644-451b-a3b2-f1366c0e09a1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:41.737156 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fc4d7-b221-4003-9a26-f4aa5c53f3e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:41.738034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fc4d7-b221-4003-9a26-f4aa5c53f3e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:41.738793 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fc4d7-b221-4003-9a26-f4aa5c53f3e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:41.739233 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fc4d7-b221-4003-9a26-f4aa5c53f3e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:41.761522 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801fc4d7-b221-4003-9a26-f4aa5c53f3e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:41.763677 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801fc4d7-b221-4003-9a26-f4aa5c53f3e1 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:41.774294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 858/1715] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:41.775680 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fc4d7-b221-4003-9a26-f4aa5c53f3e1 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:41.777509 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa48e88-c19b-4c0d-a2e9-57d21b2840af admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:41.782049 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915d78d-f17c-44e3-a52a-f6f026c1a341 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:41.789966 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915d78d-f17c-44e3-a52a-f6f026c1a341 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:41.790370 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915d78d-f17c-44e3-a52a-f6f026c1a341 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:41.790739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915d78d-f17c-44e3-a52a-f6f026c1a341 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:41.797452 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fc4d7-b221-4003-9a26-f4aa5c53f3e1 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:41.810362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6561f03-d661-4603-899e-5cb590937c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:41.811349 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6561f03-d661-4603-899e-5cb590937c3c admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:41.811637 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6561f03-d661-4603-899e-5cb590937c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:41.811912 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6561f03-d661-4603-899e-5cb590937c3c admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:41.812563 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6561f03-d661-4603-899e-5cb590937c3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:41.814530 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6561f03-d661-4603-899e-5cb590937c3c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:41.898095 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b915d78d-f17c-44e3-a52a-f6f026c1a341 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:41.899630 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b915d78d-f17c-44e3-a52a-f6f026c1a341 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:41.911366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 858/1716] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:41 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:41.912466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915d78d-f17c-44e3-a52a-f6f026c1a341 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:41.919322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 859/1717] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:32:41 2022] POST /identity/v3/users => generated 295 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:41.919322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f3518-f304-4ab6-8be4-22664131f799 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:41.919322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f3518-f304-4ab6-8be4-22664131f799 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:41.919632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f3518-f304-4ab6-8be4-22664131f799 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:41.921501 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 860/1718] 10.222.0.57 () {54 vars in 912 bytes} [Fri Sep 2 09:32:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:32:41.923760 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f3518-f304-4ab6-8be4-22664131f799 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:41.937811 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915d78d-f17c-44e3-a52a-f6f026c1a341 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:41.943704 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d963f8-8326-40a2-aeb4-0e734db5061e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:41.944456 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d963f8-8326-40a2-aeb4-0e734db5061e admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:41.944738 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d963f8-8326-40a2-aeb4-0e734db5061e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:41.945014 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d963f8-8326-40a2-aeb4-0e734db5061e admin admin] PATH_INFO: `/v3/projects/c7674da09afa4458bd89cc0fc57d448d/users/5a931551170042ee90bb9eb33a1a2838/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:41.948097 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d963f8-8326-40a2-aeb4-0e734db5061e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7674da09afa4458bd89cc0fc57d448d, user_id=5a931551170042ee90bb9eb33a1a2838, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:41.950764 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d963f8-8326-40a2-aeb4-0e734db5061e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:41.954291 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f3518-f304-4ab6-8be4-22664131f799 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:41.960310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76860b-5b2c-45b4-ba09-049d81530710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:41.960966 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76860b-5b2c-45b4-ba09-049d81530710 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:41.961249 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76860b-5b2c-45b4-ba09-049d81530710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:41.961529 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76860b-5b2c-45b4-ba09-049d81530710 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:41.962163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 859/1719] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:41 2022] PUT /identity/v3/projects/c7674da09afa4458bd89cc0fc57d448d/users/5a931551170042ee90bb9eb33a1a2838/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:41.962293 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed76860b-5b2c-45b4-ba09-049d81530710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:41.963428 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a9e51-ea69-4faf-9c7d-9ef47251f11b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:41.963715 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a9e51-ea69-4faf-9c7d-9ef47251f11b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:41.963831 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed76860b-5b2c-45b4-ba09-049d81530710 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:41.963985 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a9e51-ea69-4faf-9c7d-9ef47251f11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:41.981029 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 861/1720] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:41 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:41.981304 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31b4b7-df2e-4f04-8f0c-97aaa00495e4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:41.981556 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31b4b7-df2e-4f04-8f0c-97aaa00495e4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:41.982217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c31b4b7-df2e-4f04-8f0c-97aaa00495e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:41.984902 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-924a9e51-ea69-4faf-9c7d-9ef47251f11b None None] MFA Rules not processed for user `dfea2ee0c9e84b48ac9bafe3173d937a`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:41.986205 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-924a9e51-ea69-4faf-9c7d-9ef47251f11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:42.011280 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0c31b4b7-df2e-4f04-8f0c-97aaa00495e4 None None] MFA Rules not processed for user `5a931551170042ee90bb9eb33a1a2838`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:42.012844 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0c31b4b7-df2e-4f04-8f0c-97aaa00495e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:42.075176 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 860/1721] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Sep 2 09:32:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:42.075940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a9e51-ea69-4faf-9c7d-9ef47251f11b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:42.093745 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:42.117930 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 862/1722] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:41 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:42.119316 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31b4b7-df2e-4f04-8f0c-97aaa00495e4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:42.123072 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0c31b4b7-df2e-4f04-8f0c-97aaa00495e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:42.141959 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c31b4b7-df2e-4f04-8f0c-97aaa00495e4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:42.148399 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb270f-86ec-4cbd-9b50-4f95746e53d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:42.148399 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb270f-86ec-4cbd-9b50-4f95746e53d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:42.148399 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb270f-86ec-4cbd-9b50-4f95746e53d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:42.148399 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb270f-86ec-4cbd-9b50-4f95746e53d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:42.157839 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:42.158061 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb270f-86ec-4cbd-9b50-4f95746e53d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:42.159622 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfb270f-86ec-4cbd-9b50-4f95746e53d7 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:42.180700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 863/1723] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:42.181409 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb270f-86ec-4cbd-9b50-4f95746e53d7 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:42.217013 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a9e51-ea69-4faf-9c7d-9ef47251f11b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:42.222486 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af613ed-9e5a-4d0a-a63f-e2621dda9e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:42.225290 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af613ed-9e5a-4d0a-a63f-e2621dda9e5e admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:42.225290 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af613ed-9e5a-4d0a-a63f-e2621dda9e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:42.225290 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af613ed-9e5a-4d0a-a63f-e2621dda9e5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:42.225290 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af613ed-9e5a-4d0a-a63f-e2621dda9e5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:42.227403 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af613ed-9e5a-4d0a-a63f-e2621dda9e5e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:42.241834 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 861/1724] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:42 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:42.242385 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb270f-86ec-4cbd-9b50-4f95746e53d7 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:42.242751 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af613ed-9e5a-4d0a-a63f-e2621dda9e5e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:42.244889 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5af613ed-9e5a-4d0a-a63f-e2621dda9e5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:42.247702 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9708a94d-41d6-4edf-ae49-821b68b7a4bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:42.254223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708a94d-41d6-4edf-ae49-821b68b7a4bd service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:42.254223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708a94d-41d6-4edf-ae49-821b68b7a4bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:42.254223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9708a94d-41d6-4edf-ae49-821b68b7a4bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:42.258123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af613ed-9e5a-4d0a-a63f-e2621dda9e5e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:42.259874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9708a94d-41d6-4edf-ae49-821b68b7a4bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:42.261365 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9708a94d-41d6-4edf-ae49-821b68b7a4bd service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:42.261882 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ea9e5-b847-4bee-80a7-ccce6a5a09d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:42.262482 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ea9e5-b847-4bee-80a7-ccce6a5a09d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:42.262787 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ea9e5-b847-4bee-80a7-ccce6a5a09d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:42.263062 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81ea9e5-b847-4bee-80a7-ccce6a5a09d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:42.264836 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c81ea9e5-b847-4bee-80a7-ccce6a5a09d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:42.271477 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 864/1725] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:42 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:42.272397 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9708a94d-41d6-4edf-ae49-821b68b7a4bd service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:42.286757 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9708a94d-41d6-4edf-ae49-821b68b7a4bd service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:42.294643 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161ee8b-e3a7-46bc-83a0-ef5c54ce4d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:42.295353 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161ee8b-e3a7-46bc-83a0-ef5c54ce4d49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:42.295628 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161ee8b-e3a7-46bc-83a0-ef5c54ce4d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:42.295889 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161ee8b-e3a7-46bc-83a0-ef5c54ce4d49 admin admin] PATH_INFO: `/v3/projects/8c74b813f3a2492dab24e9b7241680f5/users/1d687452230042c8b7025f0598484628/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:42.319957 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d161ee8b-e3a7-46bc-83a0-ef5c54ce4d49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c74b813f3a2492dab24e9b7241680f5, user_id=1d687452230042c8b7025f0598484628, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:42.322335 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d161ee8b-e3a7-46bc-83a0-ef5c54ce4d49 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:42.357939 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ea9e5-b847-4bee-80a7-ccce6a5a09d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:42.360407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81ea9e5-b847-4bee-80a7-ccce6a5a09d0 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:42.374754 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 862/1726] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:42 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:42.409371 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ea9e5-b847-4bee-80a7-ccce6a5a09d0 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:42.411591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c81ea9e5-b847-4bee-80a7-ccce6a5a09d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:42.442891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 865/1727] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:42 2022] PUT /identity/v3/projects/8c74b813f3a2492dab24e9b7241680f5/users/1d687452230042c8b7025f0598484628/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:42.442891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161ee8b-e3a7-46bc-83a0-ef5c54ce4d49 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:42.535267 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81ea9e5-b847-4bee-80a7-ccce6a5a09d0 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:42.577969 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161ee8b-e3a7-46bc-83a0-ef5c54ce4d49 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:42.581782 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6e0ab-b1af-4af9-9027-2972aca7f4da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:42.582557 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6e0ab-b1af-4af9-9027-2972aca7f4da service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:42.582698 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6e0ab-b1af-4af9-9027-2972aca7f4da service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:42.583060 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6e0ab-b1af-4af9-9027-2972aca7f4da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:42.585117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5bc6e0ab-b1af-4af9-9027-2972aca7f4da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:42.624345 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526e892-3bd2-4fda-a0ba-c61845a69f51 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5177fa4d168c4c219e09c7178bbbc790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:42.629953 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526e892-3bd2-4fda-a0ba-c61845a69f51 service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:42.629953 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526e892-3bd2-4fda-a0ba-c61845a69f51 service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:42.629953 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526e892-3bd2-4fda-a0ba-c61845a69f51 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:42.629953 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6526e892-3bd2-4fda-a0ba-c61845a69f51 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:42.640696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526e892-3bd2-4fda-a0ba-c61845a69f51 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:42.642337 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6526e892-3bd2-4fda-a0ba-c61845a69f51 service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:42.652074 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc6e0ab-b1af-4af9-9027-2972aca7f4da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:42.652605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 863/1728] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:32:42.653851 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217c667-ce73-4595-9b4e-91483dfff85d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:42.654137 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217c667-ce73-4595-9b4e-91483dfff85d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:42.654414 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f217c667-ce73-4595-9b4e-91483dfff85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:42.654623 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc6e0ab-b1af-4af9-9027-2972aca7f4da service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:42.668936 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 866/1729] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:42.687680 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f217c667-ce73-4595-9b4e-91483dfff85d None None] MFA Rules not processed for user `1d687452230042c8b7025f0598484628`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:42.688916 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f217c667-ce73-4595-9b4e-91483dfff85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:42.783591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 864/1730] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:42 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:42.788444 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6e0ab-b1af-4af9-9027-2972aca7f4da service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:42.872600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6e0ab-b1af-4af9-9027-2972aca7f4da service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:42.883900 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0cabb-0e38-4100-968d-b444685232ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:42.887462 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0cabb-0e38-4100-968d-b444685232ff admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:42.887462 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0cabb-0e38-4100-968d-b444685232ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:42.887462 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e0cabb-0e38-4100-968d-b444685232ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:42.887462 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e0cabb-0e38-4100-968d-b444685232ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:42.890093 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e0cabb-0e38-4100-968d-b444685232ff admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:43.067005 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 867/1731] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:32:42 2022] POST /identity/v3/projects => generated 387 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:43.069126 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f217c667-ce73-4595-9b4e-91483dfff85d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:43.085554 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f217c667-ce73-4595-9b4e-91483dfff85d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:43.182150 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71519c50-5677-4ca0-9ff3-a7f40c3819b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:43.183339 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71519c50-5677-4ca0-9ff3-a7f40c3819b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:43.184857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71519c50-5677-4ca0-9ff3-a7f40c3819b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:43.185144 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71519c50-5677-4ca0-9ff3-a7f40c3819b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:43.185864 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71519c50-5677-4ca0-9ff3-a7f40c3819b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:43.188068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71519c50-5677-4ca0-9ff3-a7f40c3819b6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:43.326220 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0cabb-0e38-4100-968d-b444685232ff admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:43.333321 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a5e0cabb-0e38-4100-968d-b444685232ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:43.344348 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 865/1732] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:32:43 2022] POST /identity/v3/users => generated 300 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:43.351836 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71519c50-5677-4ca0-9ff3-a7f40c3819b6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:43.379026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e0cabb-0e38-4100-968d-b444685232ff admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:43.391772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71519c50-5677-4ca0-9ff3-a7f40c3819b6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:43.396296 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7921b0-fa77-46b4-ac29-afc2362a247c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:43.401419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7921b0-fa77-46b4-ac29-afc2362a247c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:43.401721 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7921b0-fa77-46b4-ac29-afc2362a247c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:43.402375 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7921b0-fa77-46b4-ac29-afc2362a247c admin admin] PATH_INFO: `/v3/users/4d632542a6e24e84ab479f1ac1eed112` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:43.406401 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7921b0-fa77-46b4-ac29-afc2362a247c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d632542a6e24e84ab479f1ac1eed112)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:43.412144 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743cc1c4-89d6-4126-885c-e6fbc4b195fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:43.413173 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743cc1c4-89d6-4126-885c-e6fbc4b195fd admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:43.413173 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743cc1c4-89d6-4126-885c-e6fbc4b195fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:43.413406 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743cc1c4-89d6-4126-885c-e6fbc4b195fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:43.413943 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743cc1c4-89d6-4126-885c-e6fbc4b195fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:43.415022 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7921b0-fa77-46b4-ac29-afc2362a247c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:43.421179 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743cc1c4-89d6-4126-885c-e6fbc4b195fd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:43.446851 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 866/1733] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:43.447774 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743cc1c4-89d6-4126-885c-e6fbc4b195fd admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:43.464723 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743cc1c4-89d6-4126-885c-e6fbc4b195fd admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:43.470650 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfaee86-2f91-4d3c-b7a2-83b8430f704b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:43.471465 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfaee86-2f91-4d3c-b7a2-83b8430f704b admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:43.471939 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfaee86-2f91-4d3c-b7a2-83b8430f704b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:43.472466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfaee86-2f91-4d3c-b7a2-83b8430f704b admin admin] PATH_INFO: `/v3/projects/823e13c964ac49fb927ed05482a6855a/users/43e08fa620ae40509215663488669304/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:43.504964 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfaee86-2f91-4d3c-b7a2-83b8430f704b admin admin] RBAC: Authorizing `identity:create_grant(project_id=823e13c964ac49fb927ed05482a6855a, user_id=43e08fa620ae40509215663488669304, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:43.512154 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfaee86-2f91-4d3c-b7a2-83b8430f704b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:43.653144 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 867/1734] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:43 2022] PUT /identity/v3/projects/823e13c964ac49fb927ed05482a6855a/users/43e08fa620ae40509215663488669304/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:43.653781 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867b703-db6a-4d91-a59c-6f8429a56cc4 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:43.654104 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867b703-db6a-4d91-a59c-6f8429a56cc4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:43.654534 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f867b703-db6a-4d91-a59c-6f8429a56cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:43.691059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:43.696587 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8a7921b0-fa77-46b4-ac29-afc2362a247c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d632542a6e24e84ab479f1ac1eed112'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:43.709048 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f867b703-db6a-4d91-a59c-6f8429a56cc4 None None] MFA Rules not processed for user `c79aaf1f72634eefb5e89c8005afe54b`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:43.710393 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f867b703-db6a-4d91-a59c-6f8429a56cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:43.796604 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8a7921b0-fa77-46b4-ac29-afc2362a247c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d632542a6e24e84ab479f1ac1eed112'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:43.797433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8a7921b0-fa77-46b4-ac29-afc2362a247c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d632542a6e24e84ab479f1ac1eed112'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:43.798196 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8a7921b0-fa77-46b4-ac29-afc2362a247c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d632542a6e24e84ab479f1ac1eed112'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:43.812958 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8a7921b0-fa77-46b4-ac29-afc2362a247c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d632542a6e24e84ab479f1ac1eed112'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:43.837189 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 868/1735] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:32:43 2022] DELETE /identity/v3/users/4d632542a6e24e84ab479f1ac1eed112 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:43.838442 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7921b0-fa77-46b4-ac29-afc2362a247c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:43.841339 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8a7921b0-fa77-46b4-ac29-afc2362a247c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:43.898158 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 868/1736] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:43.903863 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bef906-be25-458b-bfb7-8773893472ec None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:43.903863 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bef906-be25-458b-bfb7-8773893472ec None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:43.903863 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bef906-be25-458b-bfb7-8773893472ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:43.923845 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-87bef906-be25-458b-bfb7-8773893472ec None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:43.925804 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-87bef906-be25-458b-bfb7-8773893472ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:44.078942 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7921b0-fa77-46b4-ac29-afc2362a247c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:44.088038 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe06e25-3e85-4de8-aef4-67457022f970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:44.088813 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe06e25-3e85-4de8-aef4-67457022f970 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:44.089232 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe06e25-3e85-4de8-aef4-67457022f970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:44.089583 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe06e25-3e85-4de8-aef4-67457022f970 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:44.090752 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe06e25-3e85-4de8-aef4-67457022f970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:44.092945 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe06e25-3e85-4de8-aef4-67457022f970 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:44.099653 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 869/1737] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:44.109052 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bef906-be25-458b-bfb7-8773893472ec None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:44.110610 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-87bef906-be25-458b-bfb7-8773893472ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:44.112353 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 869/1738] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:43 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:44.122511 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe06e25-3e85-4de8-aef4-67457022f970 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:44.153865 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe06e25-3e85-4de8-aef4-67457022f970 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:44.178402 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23cb7ed-1c94-49ed-8f82-f686896243b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:44.179717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23cb7ed-1c94-49ed-8f82-f686896243b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:44.187033 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23cb7ed-1c94-49ed-8f82-f686896243b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:44.187769 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23cb7ed-1c94-49ed-8f82-f686896243b1 admin admin] PATH_INFO: `/v3/projects/823e13c964ac49fb927ed05482a6855a/users/43e08fa620ae40509215663488669304/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:44.193300 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23cb7ed-1c94-49ed-8f82-f686896243b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=823e13c964ac49fb927ed05482a6855a, user_id=43e08fa620ae40509215663488669304, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:44.204445 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23cb7ed-1c94-49ed-8f82-f686896243b1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:44.236225 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 870/1739] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:44 2022] PUT /identity/v3/projects/823e13c964ac49fb927ed05482a6855a/users/43e08fa620ae40509215663488669304/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:44.246935 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211e265-1f10-4b3d-95a2-159aef15f83b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:44.247567 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211e265-1f10-4b3d-95a2-159aef15f83b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:44.248519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211e265-1f10-4b3d-95a2-159aef15f83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:44.278374 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bef906-be25-458b-bfb7-8773893472ec None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:44.337842 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b211e265-1f10-4b3d-95a2-159aef15f83b None None] MFA Rules not processed for user `43e08fa620ae40509215663488669304`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:44.343862 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b211e265-1f10-4b3d-95a2-159aef15f83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:44.376436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:44.439713 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d4899e-5094-4838-9e60-a8c29b501d17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:44.440668 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4899e-5094-4838-9e60-a8c29b501d17 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:44.441174 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4899e-5094-4838-9e60-a8c29b501d17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:44.441452 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d4899e-5094-4838-9e60-a8c29b501d17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:44.562071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 871/1740] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:44 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:44.575611 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d4899e-5094-4838-9e60-a8c29b501d17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:44.577141 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d4899e-5094-4838-9e60-a8c29b501d17 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:44.577890 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211e265-1f10-4b3d-95a2-159aef15f83b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:44.590404 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b211e265-1f10-4b3d-95a2-159aef15f83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:44.593619 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 870/1741] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:44.765587 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211e265-1f10-4b3d-95a2-159aef15f83b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:44.782761 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b2a0ef-884e-4c8e-8013-89ff0b60f371 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:44.785714 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2a0ef-884e-4c8e-8013-89ff0b60f371 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:44.786174 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2a0ef-884e-4c8e-8013-89ff0b60f371 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:44.787151 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2a0ef-884e-4c8e-8013-89ff0b60f371 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:44.821700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b2a0ef-884e-4c8e-8013-89ff0b60f371 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:44.822447 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b2a0ef-884e-4c8e-8013-89ff0b60f371 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:44.861140 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 872/1742] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:44 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:45.513795 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d4899e-5094-4838-9e60-a8c29b501d17 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:45.526416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d4899e-5094-4838-9e60-a8c29b501d17 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:45.551576 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b2a0ef-884e-4c8e-8013-89ff0b60f371 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:45.553203 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-40b2a0ef-884e-4c8e-8013-89ff0b60f371 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:45.555503 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de719445-50d9-4e78-9f02-117e87a2950f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:45.556384 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de719445-50d9-4e78-9f02-117e87a2950f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:45.556815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de719445-50d9-4e78-9f02-117e87a2950f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:45.558422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de719445-50d9-4e78-9f02-117e87a2950f admin admin] PATH_INFO: `/v3/users/1d687452230042c8b7025f0598484628` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:45.561917 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de719445-50d9-4e78-9f02-117e87a2950f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d687452230042c8b7025f0598484628)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:45.567663 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de719445-50d9-4e78-9f02-117e87a2950f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:45.576909 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b2a0ef-884e-4c8e-8013-89ff0b60f371 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:45.592964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f677d3b-894d-4b15-8fd1-d8219888ed92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:45.597805 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f677d3b-894d-4b15-8fd1-d8219888ed92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:45.598375 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f677d3b-894d-4b15-8fd1-d8219888ed92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:45.599777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f677d3b-894d-4b15-8fd1-d8219888ed92 admin admin] PATH_INFO: `/v3/projects/7f784a9048784529aaf1febe6fe1322d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:45.605501 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f677d3b-894d-4b15-8fd1-d8219888ed92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f784a9048784529aaf1febe6fe1322d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:45.605501 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f677d3b-894d-4b15-8fd1-d8219888ed92 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:45.765229 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:45.787611 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5f677d3b-894d-4b15-8fd1-d8219888ed92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f784a9048784529aaf1febe6fe1322d', 'request_id': 'req-5f677d3b-894d-4b15-8fd1-d8219888ed92'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:45.794232 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5f677d3b-894d-4b15-8fd1-d8219888ed92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f784a9048784529aaf1febe6fe1322d', 'request_id': 'req-5f677d3b-894d-4b15-8fd1-d8219888ed92'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:45.801864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5f677d3b-894d-4b15-8fd1-d8219888ed92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f784a9048784529aaf1febe6fe1322d', 'request_id': 'req-5f677d3b-894d-4b15-8fd1-d8219888ed92'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:45.933773 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-de719445-50d9-4e78-9f02-117e87a2950f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d687452230042c8b7025f0598484628'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:45.947299 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5f677d3b-894d-4b15-8fd1-d8219888ed92 admin admin] The token cache is being invalidate because project 7f784a9048784529aaf1febe6fe1322d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:32:45.947865 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5f677d3b-894d-4b15-8fd1-d8219888ed92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:45.948925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5f677d3b-894d-4b15-8fd1-d8219888ed92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:45.951884 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 873/1743] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:32:45 2022] DELETE /identity/v3/projects/7f784a9048784529aaf1febe6fe1322d => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:45.956763 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4ebba-6c94-4dd2-9f61-642daeaa44cd None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:45.957502 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-de719445-50d9-4e78-9f02-117e87a2950f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d687452230042c8b7025f0598484628'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:45.959999 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4ebba-6c94-4dd2-9f61-642daeaa44cd None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:45.960662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e4ebba-6c94-4dd2-9f61-642daeaa44cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:46.004935 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c6e4ebba-6c94-4dd2-9f61-642daeaa44cd None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:46.008475 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c6e4ebba-6c94-4dd2-9f61-642daeaa44cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:46.070672 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-de719445-50d9-4e78-9f02-117e87a2950f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d687452230042c8b7025f0598484628'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:46.071518 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-de719445-50d9-4e78-9f02-117e87a2950f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d687452230042c8b7025f0598484628'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:46.092568 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-de719445-50d9-4e78-9f02-117e87a2950f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d687452230042c8b7025f0598484628'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:46.106849 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 874/1744] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:46.106849 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893d6b9-2df6-458b-81b2-b41fda42b683 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:46.106849 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893d6b9-2df6-458b-81b2-b41fda42b683 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:46.106849 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9893d6b9-2df6-458b-81b2-b41fda42b683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:46.114193 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 871/1745] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:32:45 2022] DELETE /identity/v3/users/1d687452230042c8b7025f0598484628 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:46.116311 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211f470-04ad-43aa-aebf-bd10f3d1f1b6 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:46.116761 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211f470-04ad-43aa-aebf-bd10f3d1f1b6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:46.117796 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211f470-04ad-43aa-aebf-bd10f3d1f1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:46.154861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9893d6b9-2df6-458b-81b2-b41fda42b683 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:46.156399 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9893d6b9-2df6-458b-81b2-b41fda42b683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:46.160436 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3211f470-04ad-43aa-aebf-bd10f3d1f1b6 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:46.163730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3211f470-04ad-43aa-aebf-bd10f3d1f1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:46.309803 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 875/1746] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:46.311641 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893d6b9-2df6-458b-81b2-b41fda42b683 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:46.366929 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 872/1747] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:46.368027 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211f470-04ad-43aa-aebf-bd10f3d1f1b6 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:46.368527 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9893d6b9-2df6-458b-81b2-b41fda42b683 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:46.369965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3211f470-04ad-43aa-aebf-bd10f3d1f1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:46.388321 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b0141f-9412-40c2-b6aa-191fa5912d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:46.390736 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0141f-9412-40c2-b6aa-191fa5912d7b admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:46.390736 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0141f-9412-40c2-b6aa-191fa5912d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:46.390736 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b0141f-9412-40c2-b6aa-191fa5912d7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:46.390736 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b0141f-9412-40c2-b6aa-191fa5912d7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:32:46.390736 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b0141f-9412-40c2-b6aa-191fa5912d7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:46.400384 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b0141f-9412-40c2-b6aa-191fa5912d7b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:46.437395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 876/1748] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:32:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:32:46.444775 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b0141f-9412-40c2-b6aa-191fa5912d7b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:46.476155 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211f470-04ad-43aa-aebf-bd10f3d1f1b6 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:46.501068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ddc0d-2a2c-43a7-aa12-01cd2e29b815 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:46.503065 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ddc0d-2a2c-43a7-aa12-01cd2e29b815 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:46.503357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ddc0d-2a2c-43a7-aa12-01cd2e29b815 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:46.503640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ddc0d-2a2c-43a7-aa12-01cd2e29b815 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:46.508034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b0141f-9412-40c2-b6aa-191fa5912d7b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:46.530900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ddc0d-2a2c-43a7-aa12-01cd2e29b815 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:46.531418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f0ff6-e86e-44d0-beab-25e7955ab805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:46.532374 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ddc0d-2a2c-43a7-aa12-01cd2e29b815 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:46.532644 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f0ff6-e86e-44d0-beab-25e7955ab805 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:46.533082 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f0ff6-e86e-44d0-beab-25e7955ab805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:46.533651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f0ff6-e86e-44d0-beab-25e7955ab805 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:46.534581 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f0ff6-e86e-44d0-beab-25e7955ab805 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:46.537355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f0ff6-e86e-44d0-beab-25e7955ab805 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:46.562792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 873/1749] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:46.685603 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 877/1750] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:32:46 2022] POST /identity/v3/projects => generated 373 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:32:46.686648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ddc0d-2a2c-43a7-aa12-01cd2e29b815 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:46.707095 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ddc0d-2a2c-43a7-aa12-01cd2e29b815 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:46.982147 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cece4-20ba-4657-8706-42378442968e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:46.982942 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cece4-20ba-4657-8706-42378442968e admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:46.983276 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cece4-20ba-4657-8706-42378442968e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:46.983613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cece4-20ba-4657-8706-42378442968e admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:46.984315 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3cece4-20ba-4657-8706-42378442968e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:46.987037 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3cece4-20ba-4657-8706-42378442968e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:47.043836 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f0ff6-e86e-44d0-beab-25e7955ab805 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:47.047133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-784f0ff6-e86e-44d0-beab-25e7955ab805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:47.079412 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f0ff6-e86e-44d0-beab-25e7955ab805 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:47.107228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d44231-8723-47ba-9362-3586d1caebf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:47.108455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d44231-8723-47ba-9362-3586d1caebf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:47.108739 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d44231-8723-47ba-9362-3586d1caebf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:47.109398 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d44231-8723-47ba-9362-3586d1caebf6 admin admin] PATH_INFO: `/v3/users/9d0680d3e3644bb4a52e140a12f5f54d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:47.112187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d44231-8723-47ba-9362-3586d1caebf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d0680d3e3644bb4a52e140a12f5f54d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:47.114286 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d44231-8723-47ba-9362-3586d1caebf6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:47.135196 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 874/1751] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:32:46 2022] POST /identity/v3/users => generated 287 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:47.136471 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cece4-20ba-4657-8706-42378442968e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:47.152742 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cece4-20ba-4657-8706-42378442968e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:47.164741 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8f2e0-6996-414f-9cc7-1ce71a4d49b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:47.165594 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8f2e0-6996-414f-9cc7-1ce71a4d49b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:47.166067 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8f2e0-6996-414f-9cc7-1ce71a4d49b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:47.166619 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8f2e0-6996-414f-9cc7-1ce71a4d49b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:47.167711 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad8f2e0-6996-414f-9cc7-1ce71a4d49b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:47.172985 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad8f2e0-6996-414f-9cc7-1ce71a4d49b3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:47.225964 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 875/1752] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:32:47.227219 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8f2e0-6996-414f-9cc7-1ce71a4d49b3 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:47.245331 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8f2e0-6996-414f-9cc7-1ce71a4d49b3 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:47.252387 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d11cde-15cd-47c3-abe4-f036d634f115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:47.253789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d11cde-15cd-47c3-abe4-f036d634f115 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:47.256114 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d11cde-15cd-47c3-abe4-f036d634f115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:47.256670 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d11cde-15cd-47c3-abe4-f036d634f115 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:47.259322 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d11cde-15cd-47c3-abe4-f036d634f115 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:47.260954 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d11cde-15cd-47c3-abe4-f036d634f115 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:47.298544 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 876/1753] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:47.305508 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d11cde-15cd-47c3-abe4-f036d634f115 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:47.339789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d11cde-15cd-47c3-abe4-f036d634f115 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:47.371299 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7f4e3-0c98-4e43-9b4e-b0d5a02fd0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:47.376752 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7f4e3-0c98-4e43-9b4e-b0d5a02fd0d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:47.377053 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7f4e3-0c98-4e43-9b4e-b0d5a02fd0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:47.377315 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7f4e3-0c98-4e43-9b4e-b0d5a02fd0d4 admin admin] PATH_INFO: `/v3/projects/9ccced06207840208e83ada14514c7d0/users/0603f354493a4f949a2bdf701ca1465b/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:47.408618 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-00d44231-8723-47ba-9362-3586d1caebf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d0680d3e3644bb4a52e140a12f5f54d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:47.442020 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd7f4e3-0c98-4e43-9b4e-b0d5a02fd0d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ccced06207840208e83ada14514c7d0, user_id=0603f354493a4f949a2bdf701ca1465b, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:47.445568 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd7f4e3-0c98-4e43-9b4e-b0d5a02fd0d4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:47.536797 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-00d44231-8723-47ba-9362-3586d1caebf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d0680d3e3644bb4a52e140a12f5f54d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:47.538130 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-00d44231-8723-47ba-9362-3586d1caebf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d0680d3e3644bb4a52e140a12f5f54d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:47.539521 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-00d44231-8723-47ba-9362-3586d1caebf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d0680d3e3644bb4a52e140a12f5f54d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:47.545839 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 877/1754] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:47 2022] PUT /identity/v3/projects/9ccced06207840208e83ada14514c7d0/users/0603f354493a4f949a2bdf701ca1465b/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:47.558914 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4a186-b1af-42d4-a3b6-ec3e81fd9a40 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:47.559413 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4a186-b1af-42d4-a3b6-ec3e81fd9a40 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:47.559693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4a186-b1af-42d4-a3b6-ec3e81fd9a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:47.579135 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-00d44231-8723-47ba-9362-3586d1caebf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d0680d3e3644bb4a52e140a12f5f54d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:47.613071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 878/1755] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:32:47 2022] DELETE /identity/v3/users/9d0680d3e3644bb4a52e140a12f5f54d => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:47.613260 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d44231-8723-47ba-9362-3586d1caebf6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:47.616264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-00d44231-8723-47ba-9362-3586d1caebf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:47.624255 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-06e4a186-b1af-42d4-a3b6-ec3e81fd9a40 None None] MFA Rules not processed for user `0603f354493a4f949a2bdf701ca1465b`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:47.627404 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-06e4a186-b1af-42d4-a3b6-ec3e81fd9a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:47.773299 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d44231-8723-47ba-9362-3586d1caebf6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:47.785523 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece53279-50b1-4c12-8fe4-0a5c009e82dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:47.788164 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece53279-50b1-4c12-8fe4-0a5c009e82dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:47.789137 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece53279-50b1-4c12-8fe4-0a5c009e82dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:47.791527 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece53279-50b1-4c12-8fe4-0a5c009e82dd admin admin] PATH_INFO: `/v3/projects/8c74b813f3a2492dab24e9b7241680f5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:47.793469 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:47.796847 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece53279-50b1-4c12-8fe4-0a5c009e82dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c74b813f3a2492dab24e9b7241680f5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:47.797792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 878/1756] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:47.799295 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73821bc-8963-4c43-97c6-8a12c850cfde None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:47.805064 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73821bc-8963-4c43-97c6-8a12c850cfde None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:47.805388 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73821bc-8963-4c43-97c6-8a12c850cfde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:47.806760 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece53279-50b1-4c12-8fe4-0a5c009e82dd admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:47.831400 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:47.875658 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e73821bc-8963-4c43-97c6-8a12c850cfde None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:47.880432 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e73821bc-8963-4c43-97c6-8a12c850cfde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:47.905747 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 879/1757] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:47.906584 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73821bc-8963-4c43-97c6-8a12c850cfde None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:47.912070 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e73821bc-8963-4c43-97c6-8a12c850cfde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:47.941397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73821bc-8963-4c43-97c6-8a12c850cfde None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:47.949226 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df34ad7-cfd5-4296-a7ae-14253a15a9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:47.949692 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df34ad7-cfd5-4296-a7ae-14253a15a9bb admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:47.949976 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df34ad7-cfd5-4296-a7ae-14253a15a9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:47.950249 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df34ad7-cfd5-4296-a7ae-14253a15a9bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:47.950915 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df34ad7-cfd5-4296-a7ae-14253a15a9bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:47.956824 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df34ad7-cfd5-4296-a7ae-14253a15a9bb admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:48.022167 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ece53279-50b1-4c12-8fe4-0a5c009e82dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c74b813f3a2492dab24e9b7241680f5', 'request_id': 'req-ece53279-50b1-4c12-8fe4-0a5c009e82dd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:48.028913 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ece53279-50b1-4c12-8fe4-0a5c009e82dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c74b813f3a2492dab24e9b7241680f5', 'request_id': 'req-ece53279-50b1-4c12-8fe4-0a5c009e82dd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:48.046816 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ece53279-50b1-4c12-8fe4-0a5c009e82dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c74b813f3a2492dab24e9b7241680f5', 'request_id': 'req-ece53279-50b1-4c12-8fe4-0a5c009e82dd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:48.194004 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ece53279-50b1-4c12-8fe4-0a5c009e82dd admin admin] The token cache is being invalidate because project 8c74b813f3a2492dab24e9b7241680f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:32:48.194367 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ece53279-50b1-4c12-8fe4-0a5c009e82dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:48.195048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ece53279-50b1-4c12-8fe4-0a5c009e82dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:48.199512 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 880/1758] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:32:47 2022] POST /identity/v3/projects => generated 373 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:48.199661 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 879/1759] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:32:47 2022] DELETE /identity/v3/projects/8c74b813f3a2492dab24e9b7241680f5 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:48.199750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df34ad7-cfd5-4296-a7ae-14253a15a9bb admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:48.200461 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece53279-50b1-4c12-8fe4-0a5c009e82dd admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:48.201085 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3df34ad7-cfd5-4296-a7ae-14253a15a9bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:48.201939 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ece53279-50b1-4c12-8fe4-0a5c009e82dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:48.344994 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df34ad7-cfd5-4296-a7ae-14253a15a9bb admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:48.357870 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1678ae9-171a-4e2f-955d-7883aa5229a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:48.358511 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1678ae9-171a-4e2f-955d-7883aa5229a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:48.358983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1678ae9-171a-4e2f-955d-7883aa5229a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:48.359271 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1678ae9-171a-4e2f-955d-7883aa5229a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:48.366117 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d1678ae9-171a-4e2f-955d-7883aa5229a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:48.380943 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece53279-50b1-4c12-8fe4-0a5c009e82dd admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:48.528940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1678ae9-171a-4e2f-955d-7883aa5229a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:48.530488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1678ae9-171a-4e2f-955d-7883aa5229a0 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:48.545922 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 881/1760] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:48.546686 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1678ae9-171a-4e2f-955d-7883aa5229a0 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:48.549043 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d1678ae9-171a-4e2f-955d-7883aa5229a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:48.581697 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1678ae9-171a-4e2f-955d-7883aa5229a0 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:48.601945 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3883ce-4778-490d-b8bb-fca4afb0da1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:48.602630 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3883ce-4778-490d-b8bb-fca4afb0da1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:48.602971 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3883ce-4778-490d-b8bb-fca4afb0da1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:48.603203 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3883ce-4778-490d-b8bb-fca4afb0da1e admin admin] PATH_INFO: `/v3/users/43e08fa620ae40509215663488669304` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:48.605400 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3883ce-4778-490d-b8bb-fca4afb0da1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=43e08fa620ae40509215663488669304)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:48.613567 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3883ce-4778-490d-b8bb-fca4afb0da1e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:48.646239 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cfdd76-d23b-42b2-9954-e37eacf0941d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:48.646951 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfdd76-d23b-42b2-9954-e37eacf0941d admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:48.655239 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfdd76-d23b-42b2-9954-e37eacf0941d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:48.657685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cfdd76-d23b-42b2-9954-e37eacf0941d admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:48.658421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cfdd76-d23b-42b2-9954-e37eacf0941d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:48.673168 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cfdd76-d23b-42b2-9954-e37eacf0941d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:48.818536 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 880/1761] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:32:48 2022] POST /identity/v3/users => generated 293 bytes in 618 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:32:48.824025 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cfdd76-d23b-42b2-9954-e37eacf0941d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:48.851192 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cfdd76-d23b-42b2-9954-e37eacf0941d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:48.860161 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68d306-497f-47d9-a6f0-202384d94c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:48.860964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68d306-497f-47d9-a6f0-202384d94c34 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:48.861252 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68d306-497f-47d9-a6f0-202384d94c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:48.861577 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68d306-497f-47d9-a6f0-202384d94c34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:48.862341 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a68d306-497f-47d9-a6f0-202384d94c34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:48.866576 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a68d306-497f-47d9-a6f0-202384d94c34 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:48.875409 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cc3883ce-4778-490d-b8bb-fca4afb0da1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43e08fa620ae40509215663488669304'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:48.880813 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 881/1762] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:48.885682 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68d306-497f-47d9-a6f0-202384d94c34 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:48.890218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cc3883ce-4778-490d-b8bb-fca4afb0da1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43e08fa620ae40509215663488669304'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:48.911600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68d306-497f-47d9-a6f0-202384d94c34 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:48.922664 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e3218-7993-4e12-9f05-fc4396ec27be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:48.923648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e3218-7993-4e12-9f05-fc4396ec27be admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:48.924387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e3218-7993-4e12-9f05-fc4396ec27be admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:48.924974 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e3218-7993-4e12-9f05-fc4396ec27be admin admin] PATH_INFO: `/v3/projects/6767f0a3ee32455ab100ce4b9272358a/users/04bcc80e89cb412d953b5dda5fdc04e1/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:48.947468 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49e3218-7993-4e12-9f05-fc4396ec27be admin admin] RBAC: Authorizing `identity:create_grant(project_id=6767f0a3ee32455ab100ce4b9272358a, user_id=04bcc80e89cb412d953b5dda5fdc04e1, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:48.963821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49e3218-7993-4e12-9f05-fc4396ec27be admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:48.982848 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cc3883ce-4778-490d-b8bb-fca4afb0da1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43e08fa620ae40509215663488669304'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:48.985130 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cc3883ce-4778-490d-b8bb-fca4afb0da1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43e08fa620ae40509215663488669304'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:49.022103 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cc3883ce-4778-490d-b8bb-fca4afb0da1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43e08fa620ae40509215663488669304'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:49.038687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 882/1763] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:32:48 2022] DELETE /identity/v3/users/43e08fa620ae40509215663488669304 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:49.040664 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff11252-ce9d-4e13-a143-629729dea33e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:49.041160 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff11252-ce9d-4e13-a143-629729dea33e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:49.041566 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff11252-ce9d-4e13-a143-629729dea33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:49.049307 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 882/1764] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:48 2022] PUT /identity/v3/projects/6767f0a3ee32455ab100ce4b9272358a/users/04bcc80e89cb412d953b5dda5fdc04e1/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:32:49.053970 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e3218-7993-4e12-9f05-fc4396ec27be admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:49.056323 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a49e3218-7993-4e12-9f05-fc4396ec27be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:49.232954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e3218-7993-4e12-9f05-fc4396ec27be admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:49.245356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8ff11252-ce9d-4e13-a143-629729dea33e None None] MFA Rules not processed for user `5233689a53364258b86ad2fa012d8db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:49.246368 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8ff11252-ce9d-4e13-a143-629729dea33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:49.250075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ad183-c647-432c-94f4-96a492ba6131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:49.251043 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ad183-c647-432c-94f4-96a492ba6131 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:49.251467 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ad183-c647-432c-94f4-96a492ba6131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:49.251998 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ad183-c647-432c-94f4-96a492ba6131 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:49.252815 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ad183-c647-432c-94f4-96a492ba6131 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:49.254939 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ad183-c647-432c-94f4-96a492ba6131 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:49.281872 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 883/1765] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:49 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:49.283077 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ad183-c647-432c-94f4-96a492ba6131 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:49.307316 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ad183-c647-432c-94f4-96a492ba6131 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:49.319524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0875cd-bc90-4316-811a-ceca6dbea6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:49.320761 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0875cd-bc90-4316-811a-ceca6dbea6ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:49.321186 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0875cd-bc90-4316-811a-ceca6dbea6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:49.321612 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0875cd-bc90-4316-811a-ceca6dbea6ed admin admin] PATH_INFO: `/v3/projects/6767f0a3ee32455ab100ce4b9272358a/users/04bcc80e89cb412d953b5dda5fdc04e1/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:49.326744 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0875cd-bc90-4316-811a-ceca6dbea6ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=6767f0a3ee32455ab100ce4b9272358a, user_id=04bcc80e89cb412d953b5dda5fdc04e1, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:49.331930 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0875cd-bc90-4316-811a-ceca6dbea6ed admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:49.369900 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 884/1766] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:49 2022] PUT /identity/v3/projects/6767f0a3ee32455ab100ce4b9272358a/users/04bcc80e89cb412d953b5dda5fdc04e1/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:49.373037 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a5a06-5b13-48a3-8bfc-e6b877de036c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:49.373327 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a5a06-5b13-48a3-8bfc-e6b877de036c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:49.373661 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2a5a06-5b13-48a3-8bfc-e6b877de036c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:49.404672 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 883/1767] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Sep 2 09:32:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:49.453185 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-cf2a5a06-5b13-48a3-8bfc-e6b877de036c None None] MFA Rules not processed for user `04bcc80e89cb412d953b5dda5fdc04e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:49.454502 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cf2a5a06-5b13-48a3-8bfc-e6b877de036c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:49.617275 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 885/1768] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:49 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:49.779247 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:49.794879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff11252-ce9d-4e13-a143-629729dea33e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:49.797189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8ff11252-ce9d-4e13-a143-629729dea33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:49.869315 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:49.886342 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a5a06-5b13-48a3-8bfc-e6b877de036c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:49.888170 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cf2a5a06-5b13-48a3-8bfc-e6b877de036c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:49.918601 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff11252-ce9d-4e13-a143-629729dea33e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:50.041198 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2a5a06-5b13-48a3-8bfc-e6b877de036c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:50.046812 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c1855-b07a-42a9-872c-9f307c3d2732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:50.047546 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-added05c-cb88-4a62-8d0a-be01c5dd80f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:50.047745 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c1855-b07a-42a9-872c-9f307c3d2732 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:50.047980 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c1855-b07a-42a9-872c-9f307c3d2732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:50.048288 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c1855-b07a-42a9-872c-9f307c3d2732 admin admin] PATH_INFO: `/v3/users/5f5b2ef048414faabeb7d6df013a5964` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:50.049382 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-added05c-cb88-4a62-8d0a-be01c5dd80f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:50.049816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-added05c-cb88-4a62-8d0a-be01c5dd80f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:50.049924 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c1855-b07a-42a9-872c-9f307c3d2732 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f5b2ef048414faabeb7d6df013a5964)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:50.050397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-added05c-cb88-4a62-8d0a-be01c5dd80f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:50.051931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c1855-b07a-42a9-872c-9f307c3d2732 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:50.096999 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-added05c-cb88-4a62-8d0a-be01c5dd80f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:50.100305 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-added05c-cb88-4a62-8d0a-be01c5dd80f7 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:50.132112 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 884/1769] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:49 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:32:50.155242 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-added05c-cb88-4a62-8d0a-be01c5dd80f7 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:50.155242 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-added05c-cb88-4a62-8d0a-be01c5dd80f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:50.269163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-added05c-cb88-4a62-8d0a-be01c5dd80f7 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:50.275571 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45249b73-5b0a-4d97-8c0f-680c9067be40 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30fe233a7d4c4fbea6ede233bd9ff8c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:50.276251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45249b73-5b0a-4d97-8c0f-680c9067be40 service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:50.276561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45249b73-5b0a-4d97-8c0f-680c9067be40 service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:50.276836 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45249b73-5b0a-4d97-8c0f-680c9067be40 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:50.409110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45249b73-5b0a-4d97-8c0f-680c9067be40 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:50.411486 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45249b73-5b0a-4d97-8c0f-680c9067be40 service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:50.422868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 885/1770] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Sep 2 09:32:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 09:32:50.534162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45249b73-5b0a-4d97-8c0f-680c9067be40 service swift] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:50.549650 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45249b73-5b0a-4d97-8c0f-680c9067be40 service swift] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:50.549875 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d92c1855-b07a-42a9-872c-9f307c3d2732 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f5b2ef048414faabeb7d6df013a5964'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:50.560451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf63628-3ae6-43df-9c92-9336d7cc7e24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:50.561700 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf63628-3ae6-43df-9c92-9336d7cc7e24 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:50.561827 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf63628-3ae6-43df-9c92-9336d7cc7e24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:50.562080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf63628-3ae6-43df-9c92-9336d7cc7e24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:50.566277 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bbf63628-3ae6-43df-9c92-9336d7cc7e24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:50.675429 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf63628-3ae6-43df-9c92-9336d7cc7e24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:50.675695 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf63628-3ae6-43df-9c92-9336d7cc7e24 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:50.682669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d92c1855-b07a-42a9-872c-9f307c3d2732 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f5b2ef048414faabeb7d6df013a5964'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:50.683456 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d92c1855-b07a-42a9-872c-9f307c3d2732 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f5b2ef048414faabeb7d6df013a5964'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:50.684227 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d92c1855-b07a-42a9-872c-9f307c3d2732 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f5b2ef048414faabeb7d6df013a5964'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:50.715797 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d92c1855-b07a-42a9-872c-9f307c3d2732 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f5b2ef048414faabeb7d6df013a5964'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:50.735220 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 886/1771] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:50.796230 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 886/1772] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:32:49 2022] DELETE /identity/v3/users/5f5b2ef048414faabeb7d6df013a5964 => generated 0 bytes in 910 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:32:50.801505 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac838d-76cf-4839-b465-6bdcd5e52682 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:50.803035 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac838d-76cf-4839-b465-6bdcd5e52682 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:50.803531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac838d-76cf-4839-b465-6bdcd5e52682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:50.863175 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ceac838d-76cf-4839-b465-6bdcd5e52682 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:50.865534 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ceac838d-76cf-4839-b465-6bdcd5e52682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:51.012904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 887/1773] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:32:51.050632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fed52-6241-4d58-b72a-72e49fbe7f69 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:51.051294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fed52-6241-4d58-b72a-72e49fbe7f69 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:51.051765 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019fed52-6241-4d58-b72a-72e49fbe7f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:51.105511 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-019fed52-6241-4d58-b72a-72e49fbe7f69 None None] MFA Rules not processed for user `dfea2ee0c9e84b48ac9bafe3173d937a`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:51.105511 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-019fed52-6241-4d58-b72a-72e49fbe7f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:51.112051 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac838d-76cf-4839-b465-6bdcd5e52682 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:51.119227 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ceac838d-76cf-4839-b465-6bdcd5e52682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:51.309252 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 887/1774] 10.222.0.16 () {58 vars in 1015 bytes} [Fri Sep 2 09:32:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:51.317381 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019fed52-6241-4d58-b72a-72e49fbe7f69 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:51.338610 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-019fed52-6241-4d58-b72a-72e49fbe7f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:51.390189 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019fed52-6241-4d58-b72a-72e49fbe7f69 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:51.413849 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac838d-76cf-4839-b465-6bdcd5e52682 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:51.414550 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8844c9-2275-4482-90bc-9441a49e1ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:51.414550 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8844c9-2275-4482-90bc-9441a49e1ffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:51.414550 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8844c9-2275-4482-90bc-9441a49e1ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:51.414550 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8844c9-2275-4482-90bc-9441a49e1ffd admin admin] PATH_INFO: `/v3/projects/2f0e11fc747c4619a370655d8b489cd4` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:51.418049 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ca4c1-ae1f-44d2-a382-54c6488aaa97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:51.429578 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8844c9-2275-4482-90bc-9441a49e1ffd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f0e11fc747c4619a370655d8b489cd4)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:51.430466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ca4c1-ae1f-44d2-a382-54c6488aaa97 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:51.433588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ca4c1-ae1f-44d2-a382-54c6488aaa97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:51.434290 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98ca4c1-ae1f-44d2-a382-54c6488aaa97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:51.443314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8844c9-2275-4482-90bc-9441a49e1ffd admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:51.480135 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ca4c1-ae1f-44d2-a382-54c6488aaa97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:51.487767 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98ca4c1-ae1f-44d2-a382-54c6488aaa97 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:51.502398 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:51.534477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 888/1775] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:51.535266 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ca4c1-ae1f-44d2-a382-54c6488aaa97 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:51.560095 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98ca4c1-ae1f-44d2-a382-54c6488aaa97 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:51.565564 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f47d1-c894-49d2-99ae-8acfd713f64e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:51.566550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f47d1-c894-49d2-99ae-8acfd713f64e service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:51.567121 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f47d1-c894-49d2-99ae-8acfd713f64e service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:51.567577 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f47d1-c894-49d2-99ae-8acfd713f64e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:51.578601 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f47d1-c894-49d2-99ae-8acfd713f64e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:51.581748 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f47d1-c894-49d2-99ae-8acfd713f64e service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:51.594102 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 889/1776] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:51.597080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f47d1-c894-49d2-99ae-8acfd713f64e service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:51.597332 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f53f47d1-c894-49d2-99ae-8acfd713f64e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:51.629682 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f47d1-c894-49d2-99ae-8acfd713f64e service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:51.662733 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ec8844c9-2275-4482-90bc-9441a49e1ffd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f0e11fc747c4619a370655d8b489cd4', 'request_id': 'req-ec8844c9-2275-4482-90bc-9441a49e1ffd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:51.678842 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ec8844c9-2275-4482-90bc-9441a49e1ffd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f0e11fc747c4619a370655d8b489cd4', 'request_id': 'req-ec8844c9-2275-4482-90bc-9441a49e1ffd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:51.714548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ec8844c9-2275-4482-90bc-9441a49e1ffd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f0e11fc747c4619a370655d8b489cd4', 'request_id': 'req-ec8844c9-2275-4482-90bc-9441a49e1ffd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:51.793620 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781ce64-f4bd-4531-9d63-f5ec08e5f4fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:51.794882 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781ce64-f4bd-4531-9d63-f5ec08e5f4fb service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:51.795108 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781ce64-f4bd-4531-9d63-f5ec08e5f4fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:51.795529 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781ce64-f4bd-4531-9d63-f5ec08e5f4fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:51.820533 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ec8844c9-2275-4482-90bc-9441a49e1ffd admin admin] The token cache is being invalidate because project 2f0e11fc747c4619a370655d8b489cd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:32:51.822626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ec8844c9-2275-4482-90bc-9441a49e1ffd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:51.822626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ec8844c9-2275-4482-90bc-9441a49e1ffd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:51.824455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 888/1777] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:32:51 2022] DELETE /identity/v3/projects/2f0e11fc747c4619a370655d8b489cd4 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:51.827280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781ce64-f4bd-4531-9d63-f5ec08e5f4fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:51.827940 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8844c9-2275-4482-90bc-9441a49e1ffd admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:51.830180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781ce64-f4bd-4531-9d63-f5ec08e5f4fb service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:51.835132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ec8844c9-2275-4482-90bc-9441a49e1ffd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:51.885085 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 890/1778] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:51.886741 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350dbc93-0752-43c5-bedd-6eef4f1fd978 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:51.887020 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350dbc93-0752-43c5-bedd-6eef4f1fd978 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:51.887564 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350dbc93-0752-43c5-bedd-6eef4f1fd978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:51.948195 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-350dbc93-0752-43c5-bedd-6eef4f1fd978 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:51.950419 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-350dbc93-0752-43c5-bedd-6eef4f1fd978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:52.024104 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8844c9-2275-4482-90bc-9441a49e1ffd admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:52.024104 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eafd53-7dfd-4e55-b593-99167a177827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:52.024104 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eafd53-7dfd-4e55-b593-99167a177827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:52.024104 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eafd53-7dfd-4e55-b593-99167a177827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:52.024456 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eafd53-7dfd-4e55-b593-99167a177827 admin admin] PATH_INFO: `/v3/projects/823e13c964ac49fb927ed05482a6855a` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:52.024456 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eafd53-7dfd-4e55-b593-99167a177827 admin admin] RBAC: Authorizing `identity:delete_project(project_id=823e13c964ac49fb927ed05482a6855a)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:52.024456 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eafd53-7dfd-4e55-b593-99167a177827 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:52.100536 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 891/1779] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:52.108300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d368e78-d862-4180-a89f-6837e6a19ccc None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:52.108721 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d368e78-d862-4180-a89f-6837e6a19ccc None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:52.109113 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d368e78-d862-4180-a89f-6837e6a19ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:52.150441 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2d368e78-d862-4180-a89f-6837e6a19ccc None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:52.151818 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2d368e78-d862-4180-a89f-6837e6a19ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:52.174027 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 892/1780] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:32:52.175390 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d368e78-d862-4180-a89f-6837e6a19ccc None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:52.198246 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d368e78-d862-4180-a89f-6837e6a19ccc None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:52.211641 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d77523-4503-4a49-95cf-f7d7dcb02ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:52.214038 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d77523-4503-4a49-95cf-f7d7dcb02ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:52.214328 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d77523-4503-4a49-95cf-f7d7dcb02ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:52.214608 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d77523-4503-4a49-95cf-f7d7dcb02ca7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:52.218199 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d77523-4503-4a49-95cf-f7d7dcb02ca7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:32:52.218521 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d77523-4503-4a49-95cf-f7d7dcb02ca7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:52.220216 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d77523-4503-4a49-95cf-f7d7dcb02ca7 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:52.256482 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 893/1781] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:32:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:32:52.260197 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d77523-4503-4a49-95cf-f7d7dcb02ca7 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:52.277398 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d77523-4503-4a49-95cf-f7d7dcb02ca7 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:52.279972 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-50eafd53-7dfd-4e55-b593-99167a177827 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '823e13c964ac49fb927ed05482a6855a', 'request_id': 'req-50eafd53-7dfd-4e55-b593-99167a177827'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:52.297842 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-50eafd53-7dfd-4e55-b593-99167a177827 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '823e13c964ac49fb927ed05482a6855a', 'request_id': 'req-50eafd53-7dfd-4e55-b593-99167a177827'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:52.309623 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-50eafd53-7dfd-4e55-b593-99167a177827 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '823e13c964ac49fb927ed05482a6855a', 'request_id': 'req-50eafd53-7dfd-4e55-b593-99167a177827'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:52.368840 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12185b9-babf-447e-a3c4-fc1cb15410ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:52.369774 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12185b9-babf-447e-a3c4-fc1cb15410ba admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:52.370252 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12185b9-babf-447e-a3c4-fc1cb15410ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:52.370737 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12185b9-babf-447e-a3c4-fc1cb15410ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:52.371528 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12185b9-babf-447e-a3c4-fc1cb15410ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:52.374088 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12185b9-babf-447e-a3c4-fc1cb15410ba admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:52.414180 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-50eafd53-7dfd-4e55-b593-99167a177827 admin admin] The token cache is being invalidate because project 823e13c964ac49fb927ed05482a6855a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:32:52.417830 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-50eafd53-7dfd-4e55-b593-99167a177827 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:52.417830 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-50eafd53-7dfd-4e55-b593-99167a177827 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:52.421846 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 889/1782] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:32:51 2022] DELETE /identity/v3/projects/823e13c964ac49fb927ed05482a6855a => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:52.428609 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4a28e-319f-48e8-9e81-9beb588e2914 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:52.429078 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4a28e-319f-48e8-9e81-9beb588e2914 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:52.429412 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4a28e-319f-48e8-9e81-9beb588e2914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:52.465592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2fc4a28e-319f-48e8-9e81-9beb588e2914 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:52.467610 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2fc4a28e-319f-48e8-9e81-9beb588e2914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:52.493756 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 890/1783] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:52.496662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c491f-8828-4b67-9474-7d116de355b0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:52.497022 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c491f-8828-4b67-9474-7d116de355b0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:52.497308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21c491f-8828-4b67-9474-7d116de355b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:52.517767 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 894/1784] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:32:52 2022] POST /identity/v3/projects => generated 405 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:52.519869 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12185b9-babf-447e-a3c4-fc1cb15410ba admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:52.524994 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a12185b9-babf-447e-a3c4-fc1cb15410ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:52.577361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a21c491f-8828-4b67-9474-7d116de355b0 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:52.588864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a21c491f-8828-4b67-9474-7d116de355b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:52.700093 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12185b9-babf-447e-a3c4-fc1cb15410ba admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:52.707471 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41c0b4-3351-41e2-88ec-57b0cad0754e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:52.707471 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41c0b4-3351-41e2-88ec-57b0cad0754e admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:52.707471 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41c0b4-3351-41e2-88ec-57b0cad0754e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:52.707471 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41c0b4-3351-41e2-88ec-57b0cad0754e admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:52.707957 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb41c0b4-3351-41e2-88ec-57b0cad0754e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:52.710430 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb41c0b4-3351-41e2-88ec-57b0cad0754e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:52.721225 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 891/1785] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:52.722488 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c491f-8828-4b67-9474-7d116de355b0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:52.736329 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21c491f-8828-4b67-9474-7d116de355b0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:52.745913 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b0a33-6ab6-4f23-8947-70bc635a320c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:52.746792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b0a33-6ab6-4f23-8947-70bc635a320c admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:52.747247 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b0a33-6ab6-4f23-8947-70bc635a320c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:52.747663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b0a33-6ab6-4f23-8947-70bc635a320c admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:52.748518 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b0a33-6ab6-4f23-8947-70bc635a320c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:32:52.748988 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b0a33-6ab6-4f23-8947-70bc635a320c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:52.751435 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b0a33-6ab6-4f23-8947-70bc635a320c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:52.765158 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:52.770436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 892/1786] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:32:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:32:52.779190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b0a33-6ab6-4f23-8947-70bc635a320c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:52.800054 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b0a33-6ab6-4f23-8947-70bc635a320c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:52.825051 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec1bf6-587c-431c-b504-bcd964f7795b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:52.825694 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec1bf6-587c-431c-b504-bcd964f7795b admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:52.825981 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec1bf6-587c-431c-b504-bcd964f7795b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:52.826255 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec1bf6-587c-431c-b504-bcd964f7795b admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:52.828809 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec1bf6-587c-431c-b504-bcd964f7795b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:52.832083 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec1bf6-587c-431c-b504-bcd964f7795b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:52.853606 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 895/1787] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:32:52 2022] POST /identity/v3/users => generated 303 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:52.858835 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41c0b4-3351-41e2-88ec-57b0cad0754e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:52.920564 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41c0b4-3351-41e2-88ec-57b0cad0754e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:52.941180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff6b1c-ae60-466c-923d-b571e5d0ed85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:52.945801 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff6b1c-ae60-466c-923d-b571e5d0ed85 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:52.949227 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff6b1c-ae60-466c-923d-b571e5d0ed85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:52.949227 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff6b1c-ae60-466c-923d-b571e5d0ed85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:52.949227 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff6b1c-ae60-466c-923d-b571e5d0ed85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:52.967841 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff6b1c-ae60-466c-923d-b571e5d0ed85 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:53.008252 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 896/1788] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:52 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:53.012006 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff6b1c-ae60-466c-923d-b571e5d0ed85 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:53.025188 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff6b1c-ae60-466c-923d-b571e5d0ed85 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:53.030872 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ce324-4cb6-49d3-9923-bfd7b7ac59c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:53.031761 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ce324-4cb6-49d3-9923-bfd7b7ac59c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:53.032201 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ce324-4cb6-49d3-9923-bfd7b7ac59c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:53.032846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ce324-4cb6-49d3-9923-bfd7b7ac59c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:53.033895 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211ce324-4cb6-49d3-9923-bfd7b7ac59c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:53.034952 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 893/1789] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:32:52 2022] POST /identity/v3/projects => generated 373 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:32:53.036022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211ce324-4cb6-49d3-9923-bfd7b7ac59c3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:53.046952 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec1bf6-587c-431c-b504-bcd964f7795b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:53.058770 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 897/1790] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:53.063489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ce324-4cb6-49d3-9923-bfd7b7ac59c3 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:53.109165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ce324-4cb6-49d3-9923-bfd7b7ac59c3 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:53.260253 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec1bf6-587c-431c-b504-bcd964f7795b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:53.267363 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d819ab2-9aa9-4b9a-a277-a11a81c91fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:53.267767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d819ab2-9aa9-4b9a-a277-a11a81c91fc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:53.268038 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d819ab2-9aa9-4b9a-a277-a11a81c91fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:53.268388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d819ab2-9aa9-4b9a-a277-a11a81c91fc8 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:53.269075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d819ab2-9aa9-4b9a-a277-a11a81c91fc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:53.269789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aaa849-9e59-4bd8-910c-fcab80577320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:53.271374 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aaa849-9e59-4bd8-910c-fcab80577320 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:53.271839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d819ab2-9aa9-4b9a-a277-a11a81c91fc8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:53.271951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aaa849-9e59-4bd8-910c-fcab80577320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:53.272197 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aaa849-9e59-4bd8-910c-fcab80577320 admin admin] PATH_INFO: `/v3/projects/3cb7e482622741bd9d2d86825e7d31bb/users/a2a5f591fc154bf180d0fc3df789d6b1/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:53.376912 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aaa849-9e59-4bd8-910c-fcab80577320 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cb7e482622741bd9d2d86825e7d31bb, user_id=a2a5f591fc154bf180d0fc3df789d6b1, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:53.376912 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aaa849-9e59-4bd8-910c-fcab80577320 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:53.596307 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 898/1791] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:53 2022] PUT /identity/v3/projects/3cb7e482622741bd9d2d86825e7d31bb/users/a2a5f591fc154bf180d0fc3df789d6b1/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:53.602226 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 894/1792] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:32:53 2022] POST /identity/v3/users => generated 287 bytes in 557 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:53.603272 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c07f5f-dfd8-4e3b-a9c8-a7c581c969e5 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:53.603739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c07f5f-dfd8-4e3b-a9c8-a7c581c969e5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:53.604175 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c07f5f-dfd8-4e3b-a9c8-a7c581c969e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:53.605625 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d819ab2-9aa9-4b9a-a277-a11a81c91fc8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:53.617801 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d819ab2-9aa9-4b9a-a277-a11a81c91fc8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:53.650602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-77c07f5f-dfd8-4e3b-a9c8-a7c581c969e5 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:53.652481 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-77c07f5f-dfd8-4e3b-a9c8-a7c581c969e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:53.765607 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48ad07-0963-44fa-999d-d1d3ca2e6533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:53.766428 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48ad07-0963-44fa-999d-d1d3ca2e6533 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:53.766778 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48ad07-0963-44fa-999d-d1d3ca2e6533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:53.767063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48ad07-0963-44fa-999d-d1d3ca2e6533 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:53.767769 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa48ad07-0963-44fa-999d-d1d3ca2e6533 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:53.803835 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa48ad07-0963-44fa-999d-d1d3ca2e6533 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:53.835874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 895/1793] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:53.839723 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 899/1794] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:32:53.841364 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c07f5f-dfd8-4e3b-a9c8-a7c581c969e5 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:53.858832 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c07f5f-dfd8-4e3b-a9c8-a7c581c969e5 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:53.872297 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48ad07-0963-44fa-999d-d1d3ca2e6533 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:53.875070 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04396ca-8f97-49bf-9031-d09f5e0112a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:53.877557 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04396ca-8f97-49bf-9031-d09f5e0112a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:53.878360 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fa48ad07-0963-44fa-999d-d1d3ca2e6533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:53.880034 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04396ca-8f97-49bf-9031-d09f5e0112a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:53.881549 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04396ca-8f97-49bf-9031-d09f5e0112a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:53.883472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04396ca-8f97-49bf-9031-d09f5e0112a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:53.889475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04396ca-8f97-49bf-9031-d09f5e0112a2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:53.956710 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 900/1795] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:53.965702 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04396ca-8f97-49bf-9031-d09f5e0112a2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:53.993947 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04396ca-8f97-49bf-9031-d09f5e0112a2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:53.999662 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba89f77-c2d9-4211-9701-8e9180f86283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:54.000358 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba89f77-c2d9-4211-9701-8e9180f86283 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:54.000641 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba89f77-c2d9-4211-9701-8e9180f86283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:54.000919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba89f77-c2d9-4211-9701-8e9180f86283 admin admin] PATH_INFO: `/v3/projects/fd61542abd9c462a821bd8a529357369/users/3eebca61d8084714b21ba10744a61b00/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:54.037786 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba89f77-c2d9-4211-9701-8e9180f86283 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd61542abd9c462a821bd8a529357369, user_id=3eebca61d8084714b21ba10744a61b00, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:54.040591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba89f77-c2d9-4211-9701-8e9180f86283 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:54.056837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa48ad07-0963-44fa-999d-d1d3ca2e6533 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:54.061760 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5affe4e6-919a-4d28-8ec0-75989787ebf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:54.062467 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affe4e6-919a-4d28-8ec0-75989787ebf4 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:54.062779 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affe4e6-919a-4d28-8ec0-75989787ebf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:54.063057 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affe4e6-919a-4d28-8ec0-75989787ebf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:54.073566 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5affe4e6-919a-4d28-8ec0-75989787ebf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:54.075336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5affe4e6-919a-4d28-8ec0-75989787ebf4 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:54.082276 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:54.092905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 896/1796] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:54.144862 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 901/1797] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:53 2022] PUT /identity/v3/projects/fd61542abd9c462a821bd8a529357369/users/3eebca61d8084714b21ba10744a61b00/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:54.360964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b37410-4f54-4358-bac5-eb9ff1eceee3 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:54.360964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b37410-4f54-4358-bac5-eb9ff1eceee3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:54.360964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b37410-4f54-4358-bac5-eb9ff1eceee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:54.360964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a8b37410-4f54-4358-bac5-eb9ff1eceee3 None None] MFA Rules not processed for user `3eebca61d8084714b21ba10744a61b00`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:54.360964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a8b37410-4f54-4358-bac5-eb9ff1eceee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:54.380023 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba89f77-c2d9-4211-9701-8e9180f86283 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:54.382411 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-aba89f77-c2d9-4211-9701-8e9180f86283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:54.389916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 897/1798] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:54.552650 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b37410-4f54-4358-bac5-eb9ff1eceee3 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:54.554686 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a8b37410-4f54-4358-bac5-eb9ff1eceee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:54.561049 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba89f77-c2d9-4211-9701-8e9180f86283 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:54.573788 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286efdef-cdbc-45f3-9342-8ef488ad1b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:54.575417 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286efdef-cdbc-45f3-9342-8ef488ad1b01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:54.575795 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286efdef-cdbc-45f3-9342-8ef488ad1b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:54.577501 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286efdef-cdbc-45f3-9342-8ef488ad1b01 admin admin] PATH_INFO: `/v3/projects/9c0acff1856e47cab0a33d77a65899eb` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:54.591428 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286efdef-cdbc-45f3-9342-8ef488ad1b01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c0acff1856e47cab0a33d77a65899eb)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:54.597973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286efdef-cdbc-45f3-9342-8ef488ad1b01 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:54.741016 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b37410-4f54-4358-bac5-eb9ff1eceee3 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:54.741016 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e6aa5-5608-4bc7-a310-38026b2cf664 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:54.741016 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e6aa5-5608-4bc7-a310-38026b2cf664 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:54.741016 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e6aa5-5608-4bc7-a310-38026b2cf664 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:54.743168 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e6aa5-5608-4bc7-a310-38026b2cf664 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:54.743168 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e6aa5-5608-4bc7-a310-38026b2cf664 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:54.743168 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e6aa5-5608-4bc7-a310-38026b2cf664 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:54.743168 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 898/1799] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:32:54.743168 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01767e2-a7dc-4cd6-931f-36e1bf12dca8 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:54.743168 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01767e2-a7dc-4cd6-931f-36e1bf12dca8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:54.745312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01767e2-a7dc-4cd6-931f-36e1bf12dca8 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:54.745312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 899/1800] 10.222.0.57 () {54 vars in 921 bytes} [Fri Sep 2 09:32:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:32:54.746808 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01767e2-a7dc-4cd6-931f-36e1bf12dca8 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:54.766017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01767e2-a7dc-4cd6-931f-36e1bf12dca8 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:54.778865 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14478c46-ce39-44a4-b7ec-1c11b44be18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:54.779544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14478c46-ce39-44a4-b7ec-1c11b44be18e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:54.783206 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14478c46-ce39-44a4-b7ec-1c11b44be18e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:54.783512 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14478c46-ce39-44a4-b7ec-1c11b44be18e admin admin] PATH_INFO: `/v3/users/3eebca61d8084714b21ba10744a61b00` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:54.785271 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14478c46-ce39-44a4-b7ec-1c11b44be18e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3eebca61d8084714b21ba10744a61b00)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:54.789736 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14478c46-ce39-44a4-b7ec-1c11b44be18e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:54.951215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-286efdef-cdbc-45f3-9342-8ef488ad1b01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c0acff1856e47cab0a33d77a65899eb', 'request_id': 'req-286efdef-cdbc-45f3-9342-8ef488ad1b01'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:55.044495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-286efdef-cdbc-45f3-9342-8ef488ad1b01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c0acff1856e47cab0a33d77a65899eb', 'request_id': 'req-286efdef-cdbc-45f3-9342-8ef488ad1b01'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:55.052012 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-286efdef-cdbc-45f3-9342-8ef488ad1b01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c0acff1856e47cab0a33d77a65899eb', 'request_id': 'req-286efdef-cdbc-45f3-9342-8ef488ad1b01'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:55.084818 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-286efdef-cdbc-45f3-9342-8ef488ad1b01 admin admin] The token cache is being invalidate because project 9c0acff1856e47cab0a33d77a65899eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:32:55.085136 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-286efdef-cdbc-45f3-9342-8ef488ad1b01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:55.092026 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-286efdef-cdbc-45f3-9342-8ef488ad1b01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:55.095327 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 902/1801] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:32:54 2022] DELETE /identity/v3/projects/9c0acff1856e47cab0a33d77a65899eb => generated 0 bytes in 716 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:55.099332 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d68ea-a357-417c-9b99-148d4a3e7565 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:55.099545 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d68ea-a357-417c-9b99-148d4a3e7565 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:55.101108 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d68ea-a357-417c-9b99-148d4a3e7565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:55.134470 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-14478c46-ce39-44a4-b7ec-1c11b44be18e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3eebca61d8084714b21ba10744a61b00'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:55.163030 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-be0d68ea-a357-417c-9b99-148d4a3e7565 None None] MFA Rules not processed for user `1da355c8be2d496bb08feb0fe74e04b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:55.182159 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-be0d68ea-a357-417c-9b99-148d4a3e7565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:55.357420 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 903/1802] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Sep 2 09:32:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:55.359256 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d68ea-a357-417c-9b99-148d4a3e7565 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:55.362215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-be0d68ea-a357-417c-9b99-148d4a3e7565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:55.516906 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-14478c46-ce39-44a4-b7ec-1c11b44be18e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3eebca61d8084714b21ba10744a61b00'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:55.517902 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-14478c46-ce39-44a4-b7ec-1c11b44be18e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3eebca61d8084714b21ba10744a61b00'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:55.519710 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-14478c46-ce39-44a4-b7ec-1c11b44be18e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3eebca61d8084714b21ba10744a61b00'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:55.528529 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d68ea-a357-417c-9b99-148d4a3e7565 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:55.547469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177880e-7490-4560-8b52-4f0828c194b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:55.548212 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177880e-7490-4560-8b52-4f0828c194b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:55.548631 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177880e-7490-4560-8b52-4f0828c194b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:55.549254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177880e-7490-4560-8b52-4f0828c194b8 admin admin] PATH_INFO: `/v3/users/5a931551170042ee90bb9eb33a1a2838` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:55.551296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177880e-7490-4560-8b52-4f0828c194b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a931551170042ee90bb9eb33a1a2838)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:55.554253 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177880e-7490-4560-8b52-4f0828c194b8 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:55.555400 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-14478c46-ce39-44a4-b7ec-1c11b44be18e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3eebca61d8084714b21ba10744a61b00'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:55.579244 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 900/1803] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:32:54 2022] DELETE /identity/v3/users/3eebca61d8084714b21ba10744a61b00 => generated 0 bytes in 833 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:32:55.585853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14478c46-ce39-44a4-b7ec-1c11b44be18e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:55.585853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-14478c46-ce39-44a4-b7ec-1c11b44be18e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:55.724367 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14478c46-ce39-44a4-b7ec-1c11b44be18e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:55.734173 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b45c64-e83f-4a44-a723-ec910c45b0a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:55.735444 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b45c64-e83f-4a44-a723-ec910c45b0a5 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:55.736262 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b45c64-e83f-4a44-a723-ec910c45b0a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:55.737004 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b45c64-e83f-4a44-a723-ec910c45b0a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:55.740704 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-98b45c64-e83f-4a44-a723-ec910c45b0a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:55.760500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b45c64-e83f-4a44-a723-ec910c45b0a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:55.763122 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b45c64-e83f-4a44-a723-ec910c45b0a5 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:55.781263 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 901/1804] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:55.782536 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b5b98-3361-4416-9ffb-cc740d5b8aa6 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:55.783064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b5b98-3361-4416-9ffb-cc740d5b8aa6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:55.783929 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b5b98-3361-4416-9ffb-cc740d5b8aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:55.825481 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-fc8b5b98-3361-4416-9ffb-cc740d5b8aa6 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:55.828662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fc8b5b98-3361-4416-9ffb-cc740d5b8aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:55.885913 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2177880e-7490-4560-8b52-4f0828c194b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a931551170042ee90bb9eb33a1a2838'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:55.919132 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2177880e-7490-4560-8b52-4f0828c194b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a931551170042ee90bb9eb33a1a2838'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:55.985077 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 902/1805] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:55.986983 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d43c0-6ed6-4d42-8911-4d2fc1d3205f None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:55.987431 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d43c0-6ed6-4d42-8911-4d2fc1d3205f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:55.987866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d43c0-6ed6-4d42-8911-4d2fc1d3205f None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:55.990698 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 903/1806] 10.222.0.57 () {54 vars in 921 bytes} [Fri Sep 2 09:32:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:32:55.996172 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d43c0-6ed6-4d42-8911-4d2fc1d3205f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:55.999980 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-649d43c0-6ed6-4d42-8911-4d2fc1d3205f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:56.087861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d43c0-6ed6-4d42-8911-4d2fc1d3205f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:56.091854 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a6d37-8010-4161-9817-e651f0590eb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:56.092604 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a6d37-8010-4161-9817-e651f0590eb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:56.092971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a6d37-8010-4161-9817-e651f0590eb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:56.093289 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a6d37-8010-4161-9817-e651f0590eb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:56.108066 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a6d37-8010-4161-9817-e651f0590eb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:56.109713 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a6d37-8010-4161-9817-e651f0590eb4 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:56.120864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 904/1807] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:32:56.122137 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ede05a-403c-4c3b-925a-567302cf1f36 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:56.122445 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ede05a-403c-4c3b-925a-567302cf1f36 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:56.122734 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ede05a-403c-4c3b-925a-567302cf1f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:56.158072 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-44ede05a-403c-4c3b-925a-567302cf1f36 None None] MFA Rules not processed for user `1da355c8be2d496bb08feb0fe74e04b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:56.159364 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-44ede05a-403c-4c3b-925a-567302cf1f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:56.169559 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 905/1808] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Sep 2 09:32:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:56.184698 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ede05a-403c-4c3b-925a-567302cf1f36 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:56.186738 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-44ede05a-403c-4c3b-925a-567302cf1f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:56.233731 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ede05a-403c-4c3b-925a-567302cf1f36 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:56.247657 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c4d22-fd6e-4373-a7b5-1dd8e525b323 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:56.248290 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c4d22-fd6e-4373-a7b5-1dd8e525b323 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:56.248578 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c4d22-fd6e-4373-a7b5-1dd8e525b323 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:56.248857 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3c4d22-fd6e-4373-a7b5-1dd8e525b323 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:56.287303 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c4d22-fd6e-4373-a7b5-1dd8e525b323 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:56.291594 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3c4d22-fd6e-4373-a7b5-1dd8e525b323 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:56.304581 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 906/1809] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:32:56.313294 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2177880e-7490-4560-8b52-4f0828c194b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a931551170042ee90bb9eb33a1a2838'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:56.315598 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2177880e-7490-4560-8b52-4f0828c194b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a931551170042ee90bb9eb33a1a2838'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:56.333716 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2177880e-7490-4560-8b52-4f0828c194b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a931551170042ee90bb9eb33a1a2838'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:56.344355 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 904/1810] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:32:55 2022] DELETE /identity/v3/users/5a931551170042ee90bb9eb33a1a2838 => generated 0 bytes in 986 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:56.400698 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:56.781123 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c4d22-fd6e-4373-a7b5-1dd8e525b323 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:56.783464 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ca3c4d22-fd6e-4373-a7b5-1dd8e525b323 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:56.802683 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3c4d22-fd6e-4373-a7b5-1dd8e525b323 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:56.914935 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4300e6-71e5-4ae9-8ddc-bd7b55e5f4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:56.916189 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4300e6-71e5-4ae9-8ddc-bd7b55e5f4de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:56.918926 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4300e6-71e5-4ae9-8ddc-bd7b55e5f4de admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:56.921071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4300e6-71e5-4ae9-8ddc-bd7b55e5f4de admin admin] PATH_INFO: `/v3/projects/fd61542abd9c462a821bd8a529357369` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:56.921071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4300e6-71e5-4ae9-8ddc-bd7b55e5f4de admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd61542abd9c462a821bd8a529357369)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:56.928203 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4300e6-71e5-4ae9-8ddc-bd7b55e5f4de admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:57.380073 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fd4300e6-71e5-4ae9-8ddc-bd7b55e5f4de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd61542abd9c462a821bd8a529357369', 'request_id': 'req-fd4300e6-71e5-4ae9-8ddc-bd7b55e5f4de'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:57.387105 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fd4300e6-71e5-4ae9-8ddc-bd7b55e5f4de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd61542abd9c462a821bd8a529357369', 'request_id': 'req-fd4300e6-71e5-4ae9-8ddc-bd7b55e5f4de'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:57.395717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fd4300e6-71e5-4ae9-8ddc-bd7b55e5f4de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd61542abd9c462a821bd8a529357369', 'request_id': 'req-fd4300e6-71e5-4ae9-8ddc-bd7b55e5f4de'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:57.427560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177880e-7490-4560-8b52-4f0828c194b8 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:57.427560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2177880e-7490-4560-8b52-4f0828c194b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:57.452288 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177880e-7490-4560-8b52-4f0828c194b8 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:57.571443 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e1b8b-6e45-4f6a-be53-73ffa199fa4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:57.572130 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e1b8b-6e45-4f6a-be53-73ffa199fa4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:57.572426 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e1b8b-6e45-4f6a-be53-73ffa199fa4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:57.572709 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e1b8b-6e45-4f6a-be53-73ffa199fa4e admin admin] PATH_INFO: `/v3/projects/c7674da09afa4458bd89cc0fc57d448d` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:57.574379 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e1b8b-6e45-4f6a-be53-73ffa199fa4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7674da09afa4458bd89cc0fc57d448d)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:57.576353 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e1b8b-6e45-4f6a-be53-73ffa199fa4e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:57.655963 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fd4300e6-71e5-4ae9-8ddc-bd7b55e5f4de admin admin] The token cache is being invalidate because project fd61542abd9c462a821bd8a529357369 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:32:57.656315 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fd4300e6-71e5-4ae9-8ddc-bd7b55e5f4de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:57.656917 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fd4300e6-71e5-4ae9-8ddc-bd7b55e5f4de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:57.662305 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 907/1811] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:32:56 2022] DELETE /identity/v3/projects/fd61542abd9c462a821bd8a529357369 => generated 0 bytes in 878 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:57.668410 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2a0d8-5ec6-4684-9904-c584e7df75bd None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:57.668858 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2a0d8-5ec6-4684-9904-c584e7df75bd None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:57.669269 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2a0d8-5ec6-4684-9904-c584e7df75bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:57.692658 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6eb2a0d8-5ec6-4684-9904-c584e7df75bd None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:57.694091 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6eb2a0d8-5ec6-4684-9904-c584e7df75bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:57.710864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 908/1812] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:57.712716 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b3f52-c586-49d3-a990-48f9728ad28c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:57.713124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b3f52-c586-49d3-a990-48f9728ad28c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:57.713465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8b3f52-c586-49d3-a990-48f9728ad28c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:57.739127 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1c8b3f52-c586-49d3-a990-48f9728ad28c None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:57.740564 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1c8b3f52-c586-49d3-a990-48f9728ad28c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:57.755896 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 909/1813] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:57.761046 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b3f52-c586-49d3-a990-48f9728ad28c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:57.775995 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8b3f52-c586-49d3-a990-48f9728ad28c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:57.801763 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02286cfb-7370-4fe6-af31-5e14c011b9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:57.802781 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02286cfb-7370-4fe6-af31-5e14c011b9df admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:57.803294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02286cfb-7370-4fe6-af31-5e14c011b9df admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:57.803775 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02286cfb-7370-4fe6-af31-5e14c011b9df admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:57.805009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02286cfb-7370-4fe6-af31-5e14c011b9df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:32:57.805528 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02286cfb-7370-4fe6-af31-5e14c011b9df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:57.807369 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02286cfb-7370-4fe6-af31-5e14c011b9df admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:57.830422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-973e1b8b-6e45-4f6a-be53-73ffa199fa4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7674da09afa4458bd89cc0fc57d448d', 'request_id': 'req-973e1b8b-6e45-4f6a-be53-73ffa199fa4e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:57.882669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 910/1814] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:32:57.883985 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02286cfb-7370-4fe6-af31-5e14c011b9df admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:57.899295 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-973e1b8b-6e45-4f6a-be53-73ffa199fa4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7674da09afa4458bd89cc0fc57d448d', 'request_id': 'req-973e1b8b-6e45-4f6a-be53-73ffa199fa4e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:57.911687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-973e1b8b-6e45-4f6a-be53-73ffa199fa4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7674da09afa4458bd89cc0fc57d448d', 'request_id': 'req-973e1b8b-6e45-4f6a-be53-73ffa199fa4e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:57.922041 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02286cfb-7370-4fe6-af31-5e14c011b9df admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:57.944633 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-973e1b8b-6e45-4f6a-be53-73ffa199fa4e admin admin] The token cache is being invalidate because project c7674da09afa4458bd89cc0fc57d448d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:32:57.966536 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-973e1b8b-6e45-4f6a-be53-73ffa199fa4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:57.967929 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-973e1b8b-6e45-4f6a-be53-73ffa199fa4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:32:57.970574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 905/1815] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:32:57 2022] DELETE /identity/v3/projects/c7674da09afa4458bd89cc0fc57d448d => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:57.975637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285650ac-5dc9-4a87-996c-23193aacba50 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:57.979385 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285650ac-5dc9-4a87-996c-23193aacba50 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:57.979385 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285650ac-5dc9-4a87-996c-23193aacba50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:58.058077 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-285650ac-5dc9-4a87-996c-23193aacba50 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:58.071040 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-285650ac-5dc9-4a87-996c-23193aacba50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:58.074407 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e821631-6c95-4462-a851-88d104e04e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:58.075182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e821631-6c95-4462-a851-88d104e04e8c admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:58.075707 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e821631-6c95-4462-a851-88d104e04e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:58.076197 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e821631-6c95-4462-a851-88d104e04e8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:58.077352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e821631-6c95-4462-a851-88d104e04e8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:58.081451 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e821631-6c95-4462-a851-88d104e04e8c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:58.087919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 906/1816] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:32:58.089976 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80563d43-d324-46e3-a27c-5077aacc01de None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:58.090612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80563d43-d324-46e3-a27c-5077aacc01de None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:58.091120 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80563d43-d324-46e3-a27c-5077aacc01de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:58.166295 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-80563d43-d324-46e3-a27c-5077aacc01de None None] MFA Rules not processed for user `a2a5f591fc154bf180d0fc3df789d6b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:58.167659 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-80563d43-d324-46e3-a27c-5077aacc01de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:58.239388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 911/1817] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:32:57 2022] POST /identity/v3/projects => generated 375 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:58.241470 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36613d8-63d0-4f43-8a4d-03c7cf1f53d4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:58.242420 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36613d8-63d0-4f43-8a4d-03c7cf1f53d4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:58.244171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36613d8-63d0-4f43-8a4d-03c7cf1f53d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:58.311991 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e36613d8-63d0-4f43-8a4d-03c7cf1f53d4 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:58.329860 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:58.329860 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 907/1818] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:58 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:58.329860 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80563d43-d324-46e3-a27c-5077aacc01de None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:58.337474 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e36613d8-63d0-4f43-8a4d-03c7cf1f53d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:58.350346 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-80563d43-d324-46e3-a27c-5077aacc01de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:58.382931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80563d43-d324-46e3-a27c-5077aacc01de None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:58.467657 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c98a54-2c71-4bb4-87af-20a10876e06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:58.469349 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c98a54-2c71-4bb4-87af-20a10876e06a admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:58.469841 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c98a54-2c71-4bb4-87af-20a10876e06a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:58.470276 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c98a54-2c71-4bb4-87af-20a10876e06a admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:58.471585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c98a54-2c71-4bb4-87af-20a10876e06a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:58.474873 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c98a54-2c71-4bb4-87af-20a10876e06a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:58.490753 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 912/1819] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:32:58.506908 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36613d8-63d0-4f43-8a4d-03c7cf1f53d4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:58.506908 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e36613d8-63d0-4f43-8a4d-03c7cf1f53d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:58.518765 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36613d8-63d0-4f43-8a4d-03c7cf1f53d4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:58.524428 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c30cc-82a7-459d-8ea5-0032b96b40b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:58.525093 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c30cc-82a7-459d-8ea5-0032b96b40b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:58.525480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c30cc-82a7-459d-8ea5-0032b96b40b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:58.525650 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c30cc-82a7-459d-8ea5-0032b96b40b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:58.526300 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c30cc-82a7-459d-8ea5-0032b96b40b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:58.528312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283c30cc-82a7-459d-8ea5-0032b96b40b3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:58.608700 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 908/1820] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:32:58 2022] POST /identity/v3/users => generated 288 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:58.610223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c98a54-2c71-4bb4-87af-20a10876e06a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:58.626537 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c98a54-2c71-4bb4-87af-20a10876e06a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:58.632690 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211caa98-5ecd-44d9-92eb-42e256713563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:58.633560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211caa98-5ecd-44d9-92eb-42e256713563 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:58.633816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211caa98-5ecd-44d9-92eb-42e256713563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:58.634296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211caa98-5ecd-44d9-92eb-42e256713563 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:58.635096 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211caa98-5ecd-44d9-92eb-42e256713563 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:32:58.635484 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211caa98-5ecd-44d9-92eb-42e256713563 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:58.637158 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211caa98-5ecd-44d9-92eb-42e256713563 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:58.666232 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 909/1821] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:32:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:32:58.667662 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211caa98-5ecd-44d9-92eb-42e256713563 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:58.679349 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211caa98-5ecd-44d9-92eb-42e256713563 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:58.687032 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e46250-da55-43e4-9773-2215540d00db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:58.687032 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e46250-da55-43e4-9773-2215540d00db admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:58.687032 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e46250-da55-43e4-9773-2215540d00db admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:58.687032 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e46250-da55-43e4-9773-2215540d00db admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:58.687481 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e46250-da55-43e4-9773-2215540d00db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:58.687573 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 913/1822] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:32:58 2022] POST /identity/v3/projects => generated 403 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:58.687676 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c30cc-82a7-459d-8ea5-0032b96b40b3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:58.688935 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e46250-da55-43e4-9773-2215540d00db admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:58.701826 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 910/1823] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:58.702997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e46250-da55-43e4-9773-2215540d00db admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:58.719713 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e46250-da55-43e4-9773-2215540d00db admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:58.794528 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9717469f-d007-42c4-8f89-b969f8ef25b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:58.801736 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717469f-d007-42c4-8f89-b969f8ef25b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:58.801736 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717469f-d007-42c4-8f89-b969f8ef25b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:58.801736 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717469f-d007-42c4-8f89-b969f8ef25b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:58.801736 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9717469f-d007-42c4-8f89-b969f8ef25b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:58.801736 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9717469f-d007-42c4-8f89-b969f8ef25b2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:58.803377 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c30cc-82a7-459d-8ea5-0032b96b40b3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:58.808870 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e1dbd-1315-4aa8-840d-7370bf9e8c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:58.809820 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e1dbd-1315-4aa8-840d-7370bf9e8c87 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:58.810172 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e1dbd-1315-4aa8-840d-7370bf9e8c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:58.810654 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e1dbd-1315-4aa8-840d-7370bf9e8c87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:58.811483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e1dbd-1315-4aa8-840d-7370bf9e8c87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:58.813501 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e1dbd-1315-4aa8-840d-7370bf9e8c87 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:59.003604 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 911/1824] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:32:58 2022] POST /identity/v3/users => generated 308 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:59.004532 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9717469f-d007-42c4-8f89-b969f8ef25b2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:59.004924 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 914/1825] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:32:58 2022] POST /identity/v3/projects => generated 379 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:59.008660 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e1dbd-1315-4aa8-840d-7370bf9e8c87 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:59.039571 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9717469f-d007-42c4-8f89-b969f8ef25b2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:59.106253 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e1dbd-1315-4aa8-840d-7370bf9e8c87 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:59.111969 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a51b0-6755-474f-99d3-fb3ae4cb3702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:59.112349 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a51b0-6755-474f-99d3-fb3ae4cb3702 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:59.112738 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a51b0-6755-474f-99d3-fb3ae4cb3702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:59.113234 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a51b0-6755-474f-99d3-fb3ae4cb3702 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:59.114055 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a51b0-6755-474f-99d3-fb3ae4cb3702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:59.115902 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a51b0-6755-474f-99d3-fb3ae4cb3702 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:59.132747 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 915/1826] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:59 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:59.137517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a51b0-6755-474f-99d3-fb3ae4cb3702 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:59.154841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a51b0-6755-474f-99d3-fb3ae4cb3702 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:59.159987 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764971bb-8c74-437c-8761-c5bd5595e091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:59.160803 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764971bb-8c74-437c-8761-c5bd5595e091 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:59.161083 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764971bb-8c74-437c-8761-c5bd5595e091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:59.161360 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764971bb-8c74-437c-8761-c5bd5595e091 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:59.162000 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764971bb-8c74-437c-8761-c5bd5595e091 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:59.164149 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764971bb-8c74-437c-8761-c5bd5595e091 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:59.166455 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ead94d-a2a9-4520-9758-db4713dca250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:59.167214 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ead94d-a2a9-4520-9758-db4713dca250 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:59.202436 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ead94d-a2a9-4520-9758-db4713dca250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:59.202436 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ead94d-a2a9-4520-9758-db4713dca250 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:59.202436 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ead94d-a2a9-4520-9758-db4713dca250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:59.202436 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ead94d-a2a9-4520-9758-db4713dca250 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:59.202436 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 912/1827] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:59 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:59.202436 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ead94d-a2a9-4520-9758-db4713dca250 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:59.202949 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ead94d-a2a9-4520-9758-db4713dca250 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:59.202949 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bac620-6eb5-4353-9bf1-20e4d216e3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:59.202949 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bac620-6eb5-4353-9bf1-20e4d216e3f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:59.202949 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bac620-6eb5-4353-9bf1-20e4d216e3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:59.203232 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bac620-6eb5-4353-9bf1-20e4d216e3f9 admin admin] PATH_INFO: `/v3/projects/f2f5d3036d964a95bc59ff7ac678dd55/users/25dfbde29c954d54b7c1c2e03413f2bd/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:59.218917 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bac620-6eb5-4353-9bf1-20e4d216e3f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2f5d3036d964a95bc59ff7ac678dd55, user_id=25dfbde29c954d54b7c1c2e03413f2bd, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:59.221253 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bac620-6eb5-4353-9bf1-20e4d216e3f9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:59.473832 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 916/1828] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:32:59 2022] POST /identity/v3/users => generated 290 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:32:59.473832 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764971bb-8c74-437c-8761-c5bd5595e091 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:59.482255 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 913/1829] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:59 2022] PUT /identity/v3/projects/f2f5d3036d964a95bc59ff7ac678dd55/users/25dfbde29c954d54b7c1c2e03413f2bd/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:59.483576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bac620-6eb5-4353-9bf1-20e4d216e3f9 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:59.487754 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764971bb-8c74-437c-8761-c5bd5595e091 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:59.541261 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:32:59.569591 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73417dca-e5c7-4900-9bb5-a80b3a01e03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:59.570342 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73417dca-e5c7-4900-9bb5-a80b3a01e03a admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:59.570861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73417dca-e5c7-4900-9bb5-a80b3a01e03a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:59.571152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73417dca-e5c7-4900-9bb5-a80b3a01e03a admin admin] PATH_INFO: `/v3/projects/1c7edb1b29324fa791e7ea3bccb68284/users/fe7a2ddb181647f395e4b6e1103daf96/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:59.594315 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73417dca-e5c7-4900-9bb5-a80b3a01e03a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c7edb1b29324fa791e7ea3bccb68284, user_id=fe7a2ddb181647f395e4b6e1103daf96, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:59.595674 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bac620-6eb5-4353-9bf1-20e4d216e3f9 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:59.597291 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73417dca-e5c7-4900-9bb5-a80b3a01e03a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:59.601863 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4093575-9e6e-4214-b72e-b29fa3ea16c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:59.603506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4093575-9e6e-4214-b72e-b29fa3ea16c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:59.603993 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4093575-9e6e-4214-b72e-b29fa3ea16c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:59.604392 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4093575-9e6e-4214-b72e-b29fa3ea16c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:59.605290 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4093575-9e6e-4214-b72e-b29fa3ea16c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:59.607333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4093575-9e6e-4214-b72e-b29fa3ea16c8 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:59.631694 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 914/1830] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:59 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:59.632772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4093575-9e6e-4214-b72e-b29fa3ea16c8 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:59.657713 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4093575-9e6e-4214-b72e-b29fa3ea16c8 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:59.663237 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd629a-5a0d-4c44-889c-eaae0e3ef198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:59.664101 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd629a-5a0d-4c44-889c-eaae0e3ef198 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:59.664561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd629a-5a0d-4c44-889c-eaae0e3ef198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:59.664987 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd629a-5a0d-4c44-889c-eaae0e3ef198 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:59.665875 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd629a-5a0d-4c44-889c-eaae0e3ef198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:59.667717 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfd629a-5a0d-4c44-889c-eaae0e3ef198 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:59.684437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 915/1831] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:59 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:59.687249 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd629a-5a0d-4c44-889c-eaae0e3ef198 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:59.698600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfd629a-5a0d-4c44-889c-eaae0e3ef198 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:59.703275 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f014e670-6649-4641-966c-d875b515b12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:59.703905 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f014e670-6649-4641-966c-d875b515b12c admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:59.704181 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f014e670-6649-4641-966c-d875b515b12c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:59.704480 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f014e670-6649-4641-966c-d875b515b12c admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:59.705115 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f014e670-6649-4641-966c-d875b515b12c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:59.706691 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f014e670-6649-4641-966c-d875b515b12c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:59.717402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 916/1832] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:32:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:32:59.718348 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f014e670-6649-4641-966c-d875b515b12c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:59.728227 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f014e670-6649-4641-966c-d875b515b12c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:59.734263 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af8fff-a4a2-4b53-ad7a-ead7bdd73092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:59.735005 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af8fff-a4a2-4b53-ad7a-ead7bdd73092 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:59.735294 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af8fff-a4a2-4b53-ad7a-ead7bdd73092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:59.735574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af8fff-a4a2-4b53-ad7a-ead7bdd73092 admin admin] PATH_INFO: `/v3/projects/f2f5d3036d964a95bc59ff7ac678dd55/users/25dfbde29c954d54b7c1c2e03413f2bd/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:59.737548 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af8fff-a4a2-4b53-ad7a-ead7bdd73092 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2f5d3036d964a95bc59ff7ac678dd55, user_id=25dfbde29c954d54b7c1c2e03413f2bd, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:59.740935 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af8fff-a4a2-4b53-ad7a-ead7bdd73092 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:59.751262 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 917/1833] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:59 2022] PUT /identity/v3/projects/f2f5d3036d964a95bc59ff7ac678dd55/users/25dfbde29c954d54b7c1c2e03413f2bd/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:59.752041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af8fff-a4a2-4b53-ad7a-ead7bdd73092 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:59.835370 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af8fff-a4a2-4b53-ad7a-ead7bdd73092 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:32:59.839063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 917/1834] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:59 2022] PUT /identity/v3/projects/1c7edb1b29324fa791e7ea3bccb68284/users/fe7a2ddb181647f395e4b6e1103daf96/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:32:59.851377 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8f87f-421b-4fca-bb05-9b076d96cadc None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:59.851670 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8f87f-421b-4fca-bb05-9b076d96cadc None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:59.851947 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada8f87f-421b-4fca-bb05-9b076d96cadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:59.886520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ada8f87f-421b-4fca-bb05-9b076d96cadc None None] MFA Rules not processed for user `fe7a2ddb181647f395e4b6e1103daf96`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:32:59.887770 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ada8f87f-421b-4fca-bb05-9b076d96cadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:59.896480 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a38786-bd21-484c-b975-b639b991e48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:32:59.897620 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a38786-bd21-484c-b975-b639b991e48c admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:32:59.897620 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a38786-bd21-484c-b975-b639b991e48c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:32:59.898128 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a38786-bd21-484c-b975-b639b991e48c admin admin] PATH_INFO: `/v3/projects/695391889b5647d796c4ece4569bc49e/users/f381f56a08c04b6390936af9318c1e63/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:32:59.915084 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a38786-bd21-484c-b975-b639b991e48c admin admin] RBAC: Authorizing `identity:create_grant(project_id=695391889b5647d796c4ece4569bc49e, user_id=f381f56a08c04b6390936af9318c1e63, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:32:59.917633 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a38786-bd21-484c-b975-b639b991e48c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:32:59.959584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 918/1835] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:32:59.971265 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8f87f-421b-4fca-bb05-9b076d96cadc None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:32:59.972645 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ada8f87f-421b-4fca-bb05-9b076d96cadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:32:59.990759 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 918/1836] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:32:59 2022] PUT /identity/v3/projects/695391889b5647d796c4ece4569bc49e/users/f381f56a08c04b6390936af9318c1e63/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:00.000891 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f359742-a350-4863-8132-23e50629bdff None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:00.001180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f359742-a350-4863-8132-23e50629bdff None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:00.001451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f359742-a350-4863-8132-23e50629bdff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:00.029801 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6f359742-a350-4863-8132-23e50629bdff None None] MFA Rules not processed for user `f381f56a08c04b6390936af9318c1e63`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:00.031027 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6f359742-a350-4863-8132-23e50629bdff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:00.034640 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada8f87f-421b-4fca-bb05-9b076d96cadc None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:00.109341 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7a042-00a2-462c-8ae3-27fc033c68ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:00.110035 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7a042-00a2-462c-8ae3-27fc033c68ed service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:00.110333 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7a042-00a2-462c-8ae3-27fc033c68ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:00.111067 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7a042-00a2-462c-8ae3-27fc033c68ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:00.119294 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 919/1837] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:32:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:33:00.123897 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f359742-a350-4863-8132-23e50629bdff None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:00.192696 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf7a042-00a2-462c-8ae3-27fc033c68ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:00.196493 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf7a042-00a2-462c-8ae3-27fc033c68ed service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:00.203842 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 919/1838] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:32:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:33:00.211363 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f359742-a350-4863-8132-23e50629bdff None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:00.216232 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e2531-f775-476b-9154-2477e7324311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:00.216845 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e2531-f775-476b-9154-2477e7324311 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:00.217123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e2531-f775-476b-9154-2477e7324311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:00.217396 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e2531-f775-476b-9154-2477e7324311 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:00.218025 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e2531-f775-476b-9154-2477e7324311 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:00.220301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e2531-f775-476b-9154-2477e7324311 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:00.349718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 920/1839] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:33:00 2022] POST /identity/v3/projects => generated 379 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:33:00.353943 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7a042-00a2-462c-8ae3-27fc033c68ed service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:00.370286 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7a042-00a2-462c-8ae3-27fc033c68ed service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:00.430408 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6257940f-d46c-4c56-b845-aefc334c4f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:00.431056 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257940f-d46c-4c56-b845-aefc334c4f3c admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:00.431336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257940f-d46c-4c56-b845-aefc334c4f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:00.431606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257940f-d46c-4c56-b845-aefc334c4f3c admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:00.432237 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6257940f-d46c-4c56-b845-aefc334c4f3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:00.434122 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6257940f-d46c-4c56-b845-aefc334c4f3c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:00.590545 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 920/1840] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:33:00 2022] POST /identity/v3/users => generated 296 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:00.591919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e2531-f775-476b-9154-2477e7324311 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:00.602510 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e2531-f775-476b-9154-2477e7324311 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:00.607743 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a24762-705e-4f98-b8ea-9309241dcf80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:00.608889 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a24762-705e-4f98-b8ea-9309241dcf80 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:00.618919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a24762-705e-4f98-b8ea-9309241dcf80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:00.619362 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a24762-705e-4f98-b8ea-9309241dcf80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:00.620168 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a24762-705e-4f98-b8ea-9309241dcf80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:00.623442 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a24762-705e-4f98-b8ea-9309241dcf80 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:00.644899 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 921/1841] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:00.658929 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6257940f-d46c-4c56-b845-aefc334c4f3c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:00.664654 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6257940f-d46c-4c56-b845-aefc334c4f3c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:00.669296 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f27ec-f700-4226-b18c-08e201b37a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:00.670123 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f27ec-f700-4226-b18c-08e201b37a0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:00.670391 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f27ec-f700-4226-b18c-08e201b37a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:00.670662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f27ec-f700-4226-b18c-08e201b37a0d admin admin] PATH_INFO: `/v3/projects/010fb1e641cc4788a1a4d935c87f5549/users/44aaf10f21b84b999a4759254ff8aabe/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:00.687055 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f27ec-f700-4226-b18c-08e201b37a0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=010fb1e641cc4788a1a4d935c87f5549, user_id=44aaf10f21b84b999a4759254ff8aabe, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:00.689938 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f27ec-f700-4226-b18c-08e201b37a0d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:00.771123 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 921/1842] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:00 2022] PUT /identity/v3/projects/010fb1e641cc4788a1a4d935c87f5549/users/44aaf10f21b84b999a4759254ff8aabe/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:33:00.773238 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a24762-705e-4f98-b8ea-9309241dcf80 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:00.783934 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a24762-705e-4f98-b8ea-9309241dcf80 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:00.839003 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a10b4-6f6f-40bb-8375-2d84f6563831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:00.839602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a10b4-6f6f-40bb-8375-2d84f6563831 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:00.839921 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a10b4-6f6f-40bb-8375-2d84f6563831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:00.840196 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a10b4-6f6f-40bb-8375-2d84f6563831 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:00.840829 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a10b4-6f6f-40bb-8375-2d84f6563831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:00.842402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a10b4-6f6f-40bb-8375-2d84f6563831 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:00.852703 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 922/1843] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:00.857027 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f27ec-f700-4226-b18c-08e201b37a0d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:00.867207 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f27ec-f700-4226-b18c-08e201b37a0d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:00.871740 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab49be-9fc8-4891-991b-980292f53cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:00.872389 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab49be-9fc8-4891-991b-980292f53cc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:00.872666 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab49be-9fc8-4891-991b-980292f53cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:00.872941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab49be-9fc8-4891-991b-980292f53cc6 admin admin] PATH_INFO: `/v3/projects/010fb1e641cc4788a1a4d935c87f5549/users/44aaf10f21b84b999a4759254ff8aabe/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:00.874920 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab49be-9fc8-4891-991b-980292f53cc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=010fb1e641cc4788a1a4d935c87f5549, user_id=44aaf10f21b84b999a4759254ff8aabe, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:00.877536 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab49be-9fc8-4891-991b-980292f53cc6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:00.890838 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 922/1844] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:00 2022] PUT /identity/v3/projects/010fb1e641cc4788a1a4d935c87f5549/users/44aaf10f21b84b999a4759254ff8aabe/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:00.905993 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67cc7e-cfbe-4f21-a6ae-d1b570916f03 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:00.907046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67cc7e-cfbe-4f21-a6ae-d1b570916f03 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:00.907335 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67cc7e-cfbe-4f21-a6ae-d1b570916f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:00.937115 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1a67cc7e-cfbe-4f21-a6ae-d1b570916f03 None None] MFA Rules not processed for user `44aaf10f21b84b999a4759254ff8aabe`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:00.938323 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1a67cc7e-cfbe-4f21-a6ae-d1b570916f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:00.989348 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:01.016704 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 923/1845] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:00 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:01.032018 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab49be-9fc8-4891-991b-980292f53cc6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:01.098241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab49be-9fc8-4891-991b-980292f53cc6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:01.101872 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e00ae22-d64f-4b77-9c89-a6c453b242c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:01.102484 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00ae22-d64f-4b77-9c89-a6c453b242c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:01.102845 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00ae22-d64f-4b77-9c89-a6c453b242c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:01.103067 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e00ae22-d64f-4b77-9c89-a6c453b242c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:01.112152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00ae22-d64f-4b77-9c89-a6c453b242c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:01.114134 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e00ae22-d64f-4b77-9c89-a6c453b242c0 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:01.124112 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 923/1846] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:01 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:33:01.263636 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46ba64-d151-48de-9819-d7b6b9d5ef76 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:01.263924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46ba64-d151-48de-9819-d7b6b9d5ef76 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:01.264202 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46ba64-d151-48de-9819-d7b6b9d5ef76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:01.295809 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2d46ba64-d151-48de-9819-d7b6b9d5ef76 None None] MFA Rules not processed for user `25dfbde29c954d54b7c1c2e03413f2bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:01.297080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2d46ba64-d151-48de-9819-d7b6b9d5ef76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:01.456056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 924/1847] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:01 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:02.460124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cf4a0-de33-4ed5-b884-b6e9776aff6b None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:02.461843 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cf4a0-de33-4ed5-b884-b6e9776aff6b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:02.461843 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271cf4a0-de33-4ed5-b884-b6e9776aff6b None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:02.462146 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 924/1848] 10.222.0.57 () {54 vars in 921 bytes} [Fri Sep 2 09:33:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:33:02.492407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b6158-867e-4e8a-8c7c-12a113a831cd None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:02.493012 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b6158-867e-4e8a-8c7c-12a113a831cd None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:02.493012 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542b6158-867e-4e8a-8c7c-12a113a831cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:02.512695 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-542b6158-867e-4e8a-8c7c-12a113a831cd None None] MFA Rules not processed for user `5177fa4d168c4c219e09c7178bbbc790`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:02.513940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-542b6158-867e-4e8a-8c7c-12a113a831cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:02.601988 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 925/1849] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Sep 2 09:33:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:02.607574 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3923857-8959-407b-8cc3-e7fff88a534f None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:02.607942 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3923857-8959-407b-8cc3-e7fff88a534f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:02.608311 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3923857-8959-407b-8cc3-e7fff88a534f None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:02.609740 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 925/1850] 10.222.0.57 () {54 vars in 961 bytes} [Fri Sep 2 09:33:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:33:02.614564 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddaacd1-d9f1-40e4-8a6f-c640a5fde70f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:02.614902 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddaacd1-d9f1-40e4-8a6f-c640a5fde70f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:02.615177 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddaacd1-d9f1-40e4-8a6f-c640a5fde70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:02.636256 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-bddaacd1-d9f1-40e4-8a6f-c640a5fde70f None None] MFA Rules not processed for user `5177fa4d168c4c219e09c7178bbbc790`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:02.637668 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bddaacd1-d9f1-40e4-8a6f-c640a5fde70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:02.647182 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 926/1851] 10.222.0.57 () {58 vars in 1058 bytes} [Fri Sep 2 09:33:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:02.655274 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3923857-8959-407b-8cc3-e7fff88a534f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:02.675857 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3923857-8959-407b-8cc3-e7fff88a534f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:02.683944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542b704-61a4-4d78-9127-3ecf5a8c9023 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5177fa4d168c4c219e09c7178bbbc790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:02.684694 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542b704-61a4-4d78-9127-3ecf5a8c9023 service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:02.684978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542b704-61a4-4d78-9127-3ecf5a8c9023 service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:02.685280 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f542b704-61a4-4d78-9127-3ecf5a8c9023 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:02.694746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f542b704-61a4-4d78-9127-3ecf5a8c9023 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:02.696412 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f542b704-61a4-4d78-9127-3ecf5a8c9023 service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:02.706478 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 926/1852] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:33:02.918334 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddaacd1-d9f1-40e4-8a6f-c640a5fde70f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:02.929450 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddaacd1-d9f1-40e4-8a6f-c640a5fde70f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:02.938275 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d4300-5ac9-4dce-92a2-9234fbe4aa1b tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44aaf10f21b84b999a4759254ff8aabe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010fb1e641cc4788a1a4d935c87f5549', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:02.939823 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d4300-5ac9-4dce-92a2-9234fbe4aa1b tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:02.940239 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d4300-5ac9-4dce-92a2-9234fbe4aa1b tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:02.940623 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d4300-5ac9-4dce-92a2-9234fbe4aa1b tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:02.942094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d4300-5ac9-4dce-92a2-9234fbe4aa1b tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:33:02.942491 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d4300-5ac9-4dce-92a2-9234fbe4aa1b tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:02.944272 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2d4300-5ac9-4dce-92a2-9234fbe4aa1b tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:02.958347 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 927/1853] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:33:02.962787 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542b704-61a4-4d78-9127-3ecf5a8c9023 service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:02.974002 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f542b704-61a4-4d78-9127-3ecf5a8c9023 service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:02.981637 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a0021f-ada3-46fb-a6f7-5494bb706da4 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44aaf10f21b84b999a4759254ff8aabe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010fb1e641cc4788a1a4d935c87f5549', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:02.982395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0021f-ada3-46fb-a6f7-5494bb706da4 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:02.982677 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0021f-ada3-46fb-a6f7-5494bb706da4 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:02.983147 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a0021f-ada3-46fb-a6f7-5494bb706da4 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:02.983961 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a0021f-ada3-46fb-a6f7-5494bb706da4 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:02.986225 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a0021f-ada3-46fb-a6f7-5494bb706da4 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:03.072395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 927/1854] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:33:02 2022] POST /identity/v3/projects => generated 354 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:03.076706 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d4300-5ac9-4dce-92a2-9234fbe4aa1b tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:03.089806 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d4300-5ac9-4dce-92a2-9234fbe4aa1b tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:03.181893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af2ab0a-743a-4f24-99e6-14cb7ce94179 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44aaf10f21b84b999a4759254ff8aabe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010fb1e641cc4788a1a4d935c87f5549', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:03.182651 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2ab0a-743a-4f24-99e6-14cb7ce94179 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:03.183124 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2ab0a-743a-4f24-99e6-14cb7ce94179 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:03.183478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2ab0a-743a-4f24-99e6-14cb7ce94179 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:03.184205 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af2ab0a-743a-4f24-99e6-14cb7ce94179 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:33:03.184582 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af2ab0a-743a-4f24-99e6-14cb7ce94179 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:03.186521 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af2ab0a-743a-4f24-99e6-14cb7ce94179 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:03.200310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 928/1855] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:33:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:33:03.243220 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a0021f-ada3-46fb-a6f7-5494bb706da4 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:03.253724 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a0021f-ada3-46fb-a6f7-5494bb706da4 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:03.259561 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf049dff-17b2-4e7b-b13b-e02cccf67bc3 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44aaf10f21b84b999a4759254ff8aabe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '010fb1e641cc4788a1a4d935c87f5549', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:03.260202 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf049dff-17b2-4e7b-b13b-e02cccf67bc3 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:03.260489 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf049dff-17b2-4e7b-b13b-e02cccf67bc3 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:03.260764 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf049dff-17b2-4e7b-b13b-e02cccf67bc3 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] PATH_INFO: `/v3/projects/13b131b07ab647598114de30c60c4077` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:03.261905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf049dff-17b2-4e7b-b13b-e02cccf67bc3 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] RBAC: Authorizing `identity:delete_project(project_id=13b131b07ab647598114de30c60c4077)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:03.264079 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf049dff-17b2-4e7b-b13b-e02cccf67bc3 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:03.368093 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e6804-e667-4acc-a2b7-1c287fcb9922 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:03.368383 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e6804-e667-4acc-a2b7-1c287fcb9922 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:03.368766 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e6804-e667-4acc-a2b7-1c287fcb9922 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:03.370425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 929/1856] 10.222.0.57 () {54 vars in 921 bytes} [Fri Sep 2 09:33:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 02 09:33:03.382927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0ea15-3a3e-4d04-8a2a-e9d424fd2204 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:03.383079 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cf049dff-17b2-4e7b-b13b-e02cccf67bc3 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13b131b07ab647598114de30c60c4077', 'request_id': 'req-cf049dff-17b2-4e7b-b13b-e02cccf67bc3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:03.383212 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0ea15-3a3e-4d04-8a2a-e9d424fd2204 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:03.383502 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0ea15-3a3e-4d04-8a2a-e9d424fd2204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:03.388540 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cf049dff-17b2-4e7b-b13b-e02cccf67bc3 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13b131b07ab647598114de30c60c4077', 'request_id': 'req-cf049dff-17b2-4e7b-b13b-e02cccf67bc3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:03.396469 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cf049dff-17b2-4e7b-b13b-e02cccf67bc3 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13b131b07ab647598114de30c60c4077', 'request_id': 'req-cf049dff-17b2-4e7b-b13b-e02cccf67bc3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:03.407069 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e9b0ea15-3a3e-4d04-8a2a-e9d424fd2204 None None] MFA Rules not processed for user `5177fa4d168c4c219e09c7178bbbc790`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:03.408233 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e9b0ea15-3a3e-4d04-8a2a-e9d424fd2204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:03.476104 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 930/1857] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Sep 2 09:33:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:33:03.483073 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cf049dff-17b2-4e7b-b13b-e02cccf67bc3 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] The token cache is being invalidate because project 13b131b07ab647598114de30c60c4077 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:33:03.483481 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cf049dff-17b2-4e7b-b13b-e02cccf67bc3 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:03.484044 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-cf049dff-17b2-4e7b-b13b-e02cccf67bc3 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:03.485571 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 928/1858] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:33:03 2022] DELETE /identity/v3/projects/13b131b07ab647598114de30c60c4077 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:03.487472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0ea15-3a3e-4d04-8a2a-e9d424fd2204 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:03.498206 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0ea15-3a3e-4d04-8a2a-e9d424fd2204 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:03.503628 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb0e2d-c0c0-4579-a4d6-639ce13bf678 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5177fa4d168c4c219e09c7178bbbc790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:03.505772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb0e2d-c0c0-4579-a4d6-639ce13bf678 service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:03.505772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb0e2d-c0c0-4579-a4d6-639ce13bf678 service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:03.505772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb0e2d-c0c0-4579-a4d6-639ce13bf678 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:03.506932 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7fcb0e2d-c0c0-4579-a4d6-639ce13bf678 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:03.520222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb0e2d-c0c0-4579-a4d6-639ce13bf678 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:03.521869 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb0e2d-c0c0-4579-a4d6-639ce13bf678 service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:03.527705 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:03.527705 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf049dff-17b2-4e7b-b13b-e02cccf67bc3 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:03.528826 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cf049dff-17b2-4e7b-b13b-e02cccf67bc3 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:03.540516 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 931/1859] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:33:03.595375 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf049dff-17b2-4e7b-b13b-e02cccf67bc3 tempest-VolumeQuotaClassesTest-887796273 tempest-VolumeQuotaClassesTest-887796273-project-admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:03.600010 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0a45f-859b-4c93-b1df-422c5604f63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:03.600642 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0a45f-859b-4c93-b1df-422c5604f63a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:03.600921 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0a45f-859b-4c93-b1df-422c5604f63a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:03.601194 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0a45f-859b-4c93-b1df-422c5604f63a admin admin] PATH_INFO: `/v3/users/a2a5f591fc154bf180d0fc3df789d6b1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:03.602314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd0a45f-859b-4c93-b1df-422c5604f63a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2a5f591fc154bf180d0fc3df789d6b1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:03.604253 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd0a45f-859b-4c93-b1df-422c5604f63a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:04.006108 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4bd0a45f-859b-4c93-b1df-422c5604f63a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2a5f591fc154bf180d0fc3df789d6b1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:04.074688 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4bd0a45f-859b-4c93-b1df-422c5604f63a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2a5f591fc154bf180d0fc3df789d6b1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:04.075462 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4bd0a45f-859b-4c93-b1df-422c5604f63a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2a5f591fc154bf180d0fc3df789d6b1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:04.076092 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4bd0a45f-859b-4c93-b1df-422c5604f63a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2a5f591fc154bf180d0fc3df789d6b1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:04.088596 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4bd0a45f-859b-4c93-b1df-422c5604f63a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2a5f591fc154bf180d0fc3df789d6b1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:04.100827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 929/1860] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:33:03 2022] DELETE /identity/v3/users/a2a5f591fc154bf180d0fc3df789d6b1 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:33:04.372463 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb0e2d-c0c0-4579-a4d6-639ce13bf678 service placement] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:04.373884 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7fcb0e2d-c0c0-4579-a4d6-639ce13bf678 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:04.387979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb0e2d-c0c0-4579-a4d6-639ce13bf678 service placement] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:04.428511 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0a45f-859b-4c93-b1df-422c5604f63a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:04.430186 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4bd0a45f-859b-4c93-b1df-422c5604f63a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:04.453538 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b0a4c-c38a-4a8f-a825-0e491292533a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:04.454222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b0a4c-c38a-4a8f-a825-0e491292533a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:04.454458 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b0a4c-c38a-4a8f-a825-0e491292533a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:04.454756 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b0a4c-c38a-4a8f-a825-0e491292533a admin admin] PATH_INFO: `/v3/users/f381f56a08c04b6390936af9318c1e63` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:04.456045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b0a4c-c38a-4a8f-a825-0e491292533a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f381f56a08c04b6390936af9318c1e63)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:04.457940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5b0a4c-c38a-4a8f-a825-0e491292533a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:04.505861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0a45f-859b-4c93-b1df-422c5604f63a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:04.510555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41048a7a-9ee9-4ba5-9013-d5bf5ae526f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:04.511173 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41048a7a-9ee9-4ba5-9013-d5bf5ae526f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:04.511461 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41048a7a-9ee9-4ba5-9013-d5bf5ae526f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:04.511721 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41048a7a-9ee9-4ba5-9013-d5bf5ae526f2 admin admin] PATH_INFO: `/v3/projects/3cb7e482622741bd9d2d86825e7d31bb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:04.512995 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41048a7a-9ee9-4ba5-9013-d5bf5ae526f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cb7e482622741bd9d2d86825e7d31bb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:04.514944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41048a7a-9ee9-4ba5-9013-d5bf5ae526f2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:04.749997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7d5b0a4c-c38a-4a8f-a825-0e491292533a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f381f56a08c04b6390936af9318c1e63'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:04.750469 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-41048a7a-9ee9-4ba5-9013-d5bf5ae526f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cb7e482622741bd9d2d86825e7d31bb', 'request_id': 'req-41048a7a-9ee9-4ba5-9013-d5bf5ae526f2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:04.751779 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-41048a7a-9ee9-4ba5-9013-d5bf5ae526f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cb7e482622741bd9d2d86825e7d31bb', 'request_id': 'req-41048a7a-9ee9-4ba5-9013-d5bf5ae526f2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:04.751932 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7d5b0a4c-c38a-4a8f-a825-0e491292533a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f381f56a08c04b6390936af9318c1e63'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:04.758153 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-41048a7a-9ee9-4ba5-9013-d5bf5ae526f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cb7e482622741bd9d2d86825e7d31bb', 'request_id': 'req-41048a7a-9ee9-4ba5-9013-d5bf5ae526f2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:04.826569 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7d5b0a4c-c38a-4a8f-a825-0e491292533a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f381f56a08c04b6390936af9318c1e63'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:04.827310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7d5b0a4c-c38a-4a8f-a825-0e491292533a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f381f56a08c04b6390936af9318c1e63'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:04.840708 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7d5b0a4c-c38a-4a8f-a825-0e491292533a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f381f56a08c04b6390936af9318c1e63'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:04.846858 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 932/1861] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:33:04 2022] DELETE /identity/v3/users/f381f56a08c04b6390936af9318c1e63 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:04.851802 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4671c93e-f548-4ffb-8d0b-901f2f23c940 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:04.852094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4671c93e-f548-4ffb-8d0b-901f2f23c940 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:04.852382 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4671c93e-f548-4ffb-8d0b-901f2f23c940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:04.871233 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4671c93e-f548-4ffb-8d0b-901f2f23c940 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:04.872539 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4671c93e-f548-4ffb-8d0b-901f2f23c940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:04.889068 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-41048a7a-9ee9-4ba5-9013-d5bf5ae526f2 admin admin] The token cache is being invalidate because project 3cb7e482622741bd9d2d86825e7d31bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:33:04.889385 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-41048a7a-9ee9-4ba5-9013-d5bf5ae526f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:04.890188 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-41048a7a-9ee9-4ba5-9013-d5bf5ae526f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:04.891732 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 930/1862] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:33:04 2022] DELETE /identity/v3/projects/3cb7e482622741bd9d2d86825e7d31bb => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:33:04.895991 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41048a7a-9ee9-4ba5-9013-d5bf5ae526f2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:04.903538 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-41048a7a-9ee9-4ba5-9013-d5bf5ae526f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:04.919949 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:04.944216 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 933/1863] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:33:04.956523 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4671c93e-f548-4ffb-8d0b-901f2f23c940 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:04.958812 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4671c93e-f548-4ffb-8d0b-901f2f23c940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:04.972437 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41048a7a-9ee9-4ba5-9013-d5bf5ae526f2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:04.978141 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea5d3a-ea38-484a-a0f1-cb71add14054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:04.978827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea5d3a-ea38-484a-a0f1-cb71add14054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:04.979110 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea5d3a-ea38-484a-a0f1-cb71add14054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:04.979384 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fea5d3a-ea38-484a-a0f1-cb71add14054 admin admin] PATH_INFO: `/v3/users/25dfbde29c954d54b7c1c2e03413f2bd` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:04.980518 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea5d3a-ea38-484a-a0f1-cb71add14054 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25dfbde29c954d54b7c1c2e03413f2bd)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:04.982728 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fea5d3a-ea38-484a-a0f1-cb71add14054 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:05.026290 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4671c93e-f548-4ffb-8d0b-901f2f23c940 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:05.030157 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d984f9b-9a2f-4b89-96c5-7f8905613b16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:05.030795 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d984f9b-9a2f-4b89-96c5-7f8905613b16 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:05.031075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d984f9b-9a2f-4b89-96c5-7f8905613b16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:05.031347 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d984f9b-9a2f-4b89-96c5-7f8905613b16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:05.041256 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d984f9b-9a2f-4b89-96c5-7f8905613b16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:05.046305 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d984f9b-9a2f-4b89-96c5-7f8905613b16 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:05.052341 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 934/1864] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:33:05.194213 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4fea5d3a-ea38-484a-a0f1-cb71add14054 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25dfbde29c954d54b7c1c2e03413f2bd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:05.306383 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4fea5d3a-ea38-484a-a0f1-cb71add14054 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25dfbde29c954d54b7c1c2e03413f2bd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:05.307044 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4fea5d3a-ea38-484a-a0f1-cb71add14054 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25dfbde29c954d54b7c1c2e03413f2bd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:05.307835 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4fea5d3a-ea38-484a-a0f1-cb71add14054 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25dfbde29c954d54b7c1c2e03413f2bd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:05.323565 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4fea5d3a-ea38-484a-a0f1-cb71add14054 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25dfbde29c954d54b7c1c2e03413f2bd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:05.336645 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 931/1865] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:33:04 2022] DELETE /identity/v3/users/25dfbde29c954d54b7c1c2e03413f2bd => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:33:05.502929 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d984f9b-9a2f-4b89-96c5-7f8905613b16 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:05.506166 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1d984f9b-9a2f-4b89-96c5-7f8905613b16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:05.520913 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d984f9b-9a2f-4b89-96c5-7f8905613b16 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:05.622038 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede412cd-c13b-4946-87f5-007695ee67d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:05.623073 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede412cd-c13b-4946-87f5-007695ee67d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:05.623207 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede412cd-c13b-4946-87f5-007695ee67d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:05.623430 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede412cd-c13b-4946-87f5-007695ee67d0 admin admin] PATH_INFO: `/v3/projects/695391889b5647d796c4ece4569bc49e` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:05.624665 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede412cd-c13b-4946-87f5-007695ee67d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=695391889b5647d796c4ece4569bc49e)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:05.626624 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede412cd-c13b-4946-87f5-007695ee67d0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:05.745649 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ede412cd-c13b-4946-87f5-007695ee67d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '695391889b5647d796c4ece4569bc49e', 'request_id': 'req-ede412cd-c13b-4946-87f5-007695ee67d0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:05.850356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ede412cd-c13b-4946-87f5-007695ee67d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '695391889b5647d796c4ece4569bc49e', 'request_id': 'req-ede412cd-c13b-4946-87f5-007695ee67d0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:05.859864 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ede412cd-c13b-4946-87f5-007695ee67d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '695391889b5647d796c4ece4569bc49e', 'request_id': 'req-ede412cd-c13b-4946-87f5-007695ee67d0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:05.879112 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ede412cd-c13b-4946-87f5-007695ee67d0 admin admin] The token cache is being invalidate because project 695391889b5647d796c4ece4569bc49e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:33:05.879883 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ede412cd-c13b-4946-87f5-007695ee67d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:05.885473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ede412cd-c13b-4946-87f5-007695ee67d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:05.890874 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 935/1866] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:33:05 2022] DELETE /identity/v3/projects/695391889b5647d796c4ece4569bc49e => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:05.891960 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea5d3a-ea38-484a-a0f1-cb71add14054 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:05.893421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4fea5d3a-ea38-484a-a0f1-cb71add14054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:05.923995 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fea5d3a-ea38-484a-a0f1-cb71add14054 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:06.027513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede412cd-c13b-4946-87f5-007695ee67d0 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:06.029462 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ede412cd-c13b-4946-87f5-007695ee67d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:06.059055 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd43ac-546c-4cc9-9515-670dbcc7f275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:06.060635 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd43ac-546c-4cc9-9515-670dbcc7f275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:06.063150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd43ac-546c-4cc9-9515-670dbcc7f275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:06.063544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd43ac-546c-4cc9-9515-670dbcc7f275 admin admin] PATH_INFO: `/v3/users/44aaf10f21b84b999a4759254ff8aabe` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:06.064958 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd43ac-546c-4cc9-9515-670dbcc7f275 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44aaf10f21b84b999a4759254ff8aabe)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:06.069794 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd43ac-546c-4cc9-9515-670dbcc7f275 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:06.147512 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=83,keystone:INSERT=2,keystone:DELETE=15 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:06.147599 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede412cd-c13b-4946-87f5-007695ee67d0 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:06.150929 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493838cc-4a2d-46a3-ae62-e136feffed2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:06.151571 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493838cc-4a2d-46a3-ae62-e136feffed2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:06.151854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493838cc-4a2d-46a3-ae62-e136feffed2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:06.152129 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493838cc-4a2d-46a3-ae62-e136feffed2f admin admin] PATH_INFO: `/v3/projects/f2f5d3036d964a95bc59ff7ac678dd55` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:06.153319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493838cc-4a2d-46a3-ae62-e136feffed2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2f5d3036d964a95bc59ff7ac678dd55)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:06.155290 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493838cc-4a2d-46a3-ae62-e136feffed2f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:06.353377 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-65cd43ac-546c-4cc9-9515-670dbcc7f275 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44aaf10f21b84b999a4759254ff8aabe'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:06.371301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-493838cc-4a2d-46a3-ae62-e136feffed2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2f5d3036d964a95bc59ff7ac678dd55', 'request_id': 'req-493838cc-4a2d-46a3-ae62-e136feffed2f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:06.473437 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-65cd43ac-546c-4cc9-9515-670dbcc7f275 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44aaf10f21b84b999a4759254ff8aabe'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:06.474129 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-65cd43ac-546c-4cc9-9515-670dbcc7f275 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44aaf10f21b84b999a4759254ff8aabe'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:06.474847 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-65cd43ac-546c-4cc9-9515-670dbcc7f275 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44aaf10f21b84b999a4759254ff8aabe'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:06.499164 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-65cd43ac-546c-4cc9-9515-670dbcc7f275 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44aaf10f21b84b999a4759254ff8aabe'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:06.524701 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 932/1867] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:33:05 2022] DELETE /identity/v3/users/44aaf10f21b84b999a4759254ff8aabe => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:06.537301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:06.586376 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-493838cc-4a2d-46a3-ae62-e136feffed2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2f5d3036d964a95bc59ff7ac678dd55', 'request_id': 'req-493838cc-4a2d-46a3-ae62-e136feffed2f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:06.594605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-493838cc-4a2d-46a3-ae62-e136feffed2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2f5d3036d964a95bc59ff7ac678dd55', 'request_id': 'req-493838cc-4a2d-46a3-ae62-e136feffed2f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:06.614567 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-493838cc-4a2d-46a3-ae62-e136feffed2f admin admin] The token cache is being invalidate because project f2f5d3036d964a95bc59ff7ac678dd55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:33:06.618590 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-493838cc-4a2d-46a3-ae62-e136feffed2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:06.618590 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-493838cc-4a2d-46a3-ae62-e136feffed2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:06.621475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 936/1868] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:33:06 2022] DELETE /identity/v3/projects/f2f5d3036d964a95bc59ff7ac678dd55 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:06.626562 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a0482-6bc4-4834-9187-a334a73b2ced None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:06.627867 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a0482-6bc4-4834-9187-a334a73b2ced None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:06.628447 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a0482-6bc4-4834-9187-a334a73b2ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:06.671033 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8d9a0482-6bc4-4834-9187-a334a73b2ced None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:06.673542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8d9a0482-6bc4-4834-9187-a334a73b2ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:06.797533 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 933/1869] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:06.797862 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24bff69-2f70-4a34-b9d6-ad33011faa9b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:06.797862 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24bff69-2f70-4a34-b9d6-ad33011faa9b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:06.797862 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24bff69-2f70-4a34-b9d6-ad33011faa9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:06.814930 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e24bff69-2f70-4a34-b9d6-ad33011faa9b None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:06.816147 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e24bff69-2f70-4a34-b9d6-ad33011faa9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:06.824792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 937/1870] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:06.829871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a0482-6bc4-4834-9187-a334a73b2ced None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:06.841357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a0482-6bc4-4834-9187-a334a73b2ced None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:06.857815 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b6c2f-e41b-401d-a941-1d6f0fe205c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:06.858737 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b6c2f-e41b-401d-a941-1d6f0fe205c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:06.859180 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b6c2f-e41b-401d-a941-1d6f0fe205c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:06.859446 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2b6c2f-e41b-401d-a941-1d6f0fe205c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:06.860560 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2b6c2f-e41b-401d-a941-1d6f0fe205c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:33:06.861412 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2b6c2f-e41b-401d-a941-1d6f0fe205c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:06.863427 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2b6c2f-e41b-401d-a941-1d6f0fe205c1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:06.881981 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 934/1871] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:33:06.886372 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24bff69-2f70-4a34-b9d6-ad33011faa9b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:06.897235 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24bff69-2f70-4a34-b9d6-ad33011faa9b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:06.903744 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d8c36d-c965-4e05-a2c2-0fa396a1c411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:06.904741 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8c36d-c965-4e05-a2c2-0fa396a1c411 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:06.905220 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8c36d-c965-4e05-a2c2-0fa396a1c411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:06.905674 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8c36d-c965-4e05-a2c2-0fa396a1c411 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:06.906589 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d8c36d-c965-4e05-a2c2-0fa396a1c411 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:06.909375 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d8c36d-c965-4e05-a2c2-0fa396a1c411 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:07.061333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 938/1872] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:33:06 2022] POST /identity/v3/projects => generated 365 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:07.061736 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b6c2f-e41b-401d-a941-1d6f0fe205c1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:07.077566 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2b6c2f-e41b-401d-a941-1d6f0fe205c1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:07.154407 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746bf3d-15ca-43c6-a26d-3162d53babb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:07.155304 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746bf3d-15ca-43c6-a26d-3162d53babb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:07.155594 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746bf3d-15ca-43c6-a26d-3162d53babb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:07.156037 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746bf3d-15ca-43c6-a26d-3162d53babb7 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:07.157512 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746bf3d-15ca-43c6-a26d-3162d53babb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:07.160997 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746bf3d-15ca-43c6-a26d-3162d53babb7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:07.251095 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d8c36d-c965-4e05-a2c2-0fa396a1c411 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:07.252883 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-44d8c36d-c965-4e05-a2c2-0fa396a1c411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:07.264224 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 935/1873] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:33:07 2022] POST /identity/v3/users => generated 283 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:07.273329 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746bf3d-15ca-43c6-a26d-3162d53babb7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:07.279668 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d8c36d-c965-4e05-a2c2-0fa396a1c411 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:07.283346 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746bf3d-15ca-43c6-a26d-3162d53babb7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:07.290596 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bf35eb-e4e2-4540-9835-3fcc4ed3a775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:07.291549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf35eb-e4e2-4540-9835-3fcc4ed3a775 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:07.291961 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf35eb-e4e2-4540-9835-3fcc4ed3a775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:07.292245 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bf35eb-e4e2-4540-9835-3fcc4ed3a775 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:07.293303 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bf35eb-e4e2-4540-9835-3fcc4ed3a775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:07.294957 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bf35eb-e4e2-4540-9835-3fcc4ed3a775 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:07.304191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf8890b-7257-4354-8c7b-ab352a1ee58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:07.304191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf8890b-7257-4354-8c7b-ab352a1ee58e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:07.304191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf8890b-7257-4354-8c7b-ab352a1ee58e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:07.304191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf8890b-7257-4354-8c7b-ab352a1ee58e admin admin] PATH_INFO: `/v3/projects/010fb1e641cc4788a1a4d935c87f5549` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:07.304592 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf8890b-7257-4354-8c7b-ab352a1ee58e admin admin] RBAC: Authorizing `identity:delete_project(project_id=010fb1e641cc4788a1a4d935c87f5549)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:07.305137 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf8890b-7257-4354-8c7b-ab352a1ee58e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:07.307860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 936/1874] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:33:07.311814 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bf35eb-e4e2-4540-9835-3fcc4ed3a775 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:07.322802 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bf35eb-e4e2-4540-9835-3fcc4ed3a775 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:07.327939 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1ca6a-099b-4f31-b657-efc3a922ee05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:07.328628 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1ca6a-099b-4f31-b657-efc3a922ee05 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:07.329102 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1ca6a-099b-4f31-b657-efc3a922ee05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:07.329321 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1ca6a-099b-4f31-b657-efc3a922ee05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:07.332537 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee1ca6a-099b-4f31-b657-efc3a922ee05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:07.332537 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee1ca6a-099b-4f31-b657-efc3a922ee05 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:07.343429 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 937/1875] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:33:07.348187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1ca6a-099b-4f31-b657-efc3a922ee05 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:07.358924 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1ca6a-099b-4f31-b657-efc3a922ee05 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:07.364360 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f6b247-63b7-4154-b31b-a56eaa1299a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:07.365015 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6b247-63b7-4154-b31b-a56eaa1299a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:07.365297 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6b247-63b7-4154-b31b-a56eaa1299a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:07.365636 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6b247-63b7-4154-b31b-a56eaa1299a6 admin admin] PATH_INFO: `/v3/projects/a08212d9edd84d86929d313237a29697/users/c463ee48d3d04c698a4c465f5453ddad/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:07.380392 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f6b247-63b7-4154-b31b-a56eaa1299a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a08212d9edd84d86929d313237a29697, user_id=c463ee48d3d04c698a4c465f5453ddad, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:07.383167 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f6b247-63b7-4154-b31b-a56eaa1299a6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:07.539245 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 938/1876] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:07 2022] PUT /identity/v3/projects/a08212d9edd84d86929d313237a29697/users/c463ee48d3d04c698a4c465f5453ddad/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:33:07.547817 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3cf8890b-7257-4354-8c7b-ab352a1ee58e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '010fb1e641cc4788a1a4d935c87f5549', 'request_id': 'req-3cf8890b-7257-4354-8c7b-ab352a1ee58e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:07.551291 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e324dda-e458-49d5-bfcf-70a0bd769d0c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:07.551580 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e324dda-e458-49d5-bfcf-70a0bd769d0c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:07.551852 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e324dda-e458-49d5-bfcf-70a0bd769d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:07.581151 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9e324dda-e458-49d5-bfcf-70a0bd769d0c None None] MFA Rules not processed for user `c463ee48d3d04c698a4c465f5453ddad`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:07.582560 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9e324dda-e458-49d5-bfcf-70a0bd769d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:07.648045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 939/1877] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:33:07.652689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e324dda-e458-49d5-bfcf-70a0bd769d0c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:07.668903 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3cf8890b-7257-4354-8c7b-ab352a1ee58e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '010fb1e641cc4788a1a4d935c87f5549', 'request_id': 'req-3cf8890b-7257-4354-8c7b-ab352a1ee58e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:07.676183 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3cf8890b-7257-4354-8c7b-ab352a1ee58e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '010fb1e641cc4788a1a4d935c87f5549', 'request_id': 'req-3cf8890b-7257-4354-8c7b-ab352a1ee58e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:07.690587 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3cf8890b-7257-4354-8c7b-ab352a1ee58e admin admin] The token cache is being invalidate because project 010fb1e641cc4788a1a4d935c87f5549 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:33:07.690983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3cf8890b-7257-4354-8c7b-ab352a1ee58e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:07.691729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3cf8890b-7257-4354-8c7b-ab352a1ee58e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:07.693500 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 939/1878] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:33:07 2022] DELETE /identity/v3/projects/010fb1e641cc4788a1a4d935c87f5549 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:07.698243 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6206d0-8528-4581-969e-cad5ddf5f4d3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:07.698780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6206d0-8528-4581-969e-cad5ddf5f4d3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:07.699127 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6206d0-8528-4581-969e-cad5ddf5f4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:07.721871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6e6206d0-8528-4581-969e-cad5ddf5f4d3 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:07.723867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6e6206d0-8528-4581-969e-cad5ddf5f4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:07.728742 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e324dda-e458-49d5-bfcf-70a0bd769d0c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:07.733704 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148dc57-138f-45b2-ae84-694af828c2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:07.734390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148dc57-138f-45b2-ae84-694af828c2fb admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:07.734799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148dc57-138f-45b2-ae84-694af828c2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:07.734926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 940/1879] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:33:07.735235 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2148dc57-138f-45b2-ae84-694af828c2fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:07.736047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2148dc57-138f-45b2-ae84-694af828c2fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:07.737905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2148dc57-138f-45b2-ae84-694af828c2fb admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:07.742518 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb7d5b-f79a-4d0f-874f-788bda82722c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:07.743046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb7d5b-f79a-4d0f-874f-788bda82722c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:07.743331 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb7d5b-f79a-4d0f-874f-788bda82722c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:07.762642 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-47cb7d5b-f79a-4d0f-874f-788bda82722c None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:07.763874 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-47cb7d5b-f79a-4d0f-874f-788bda82722c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:07.771980 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 941/1880] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:33:07.776049 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb7d5b-f79a-4d0f-874f-788bda82722c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:07.784612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb7d5b-f79a-4d0f-874f-788bda82722c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:07.789660 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffda832-0b8b-4e0e-93a9-2cada0965d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:07.790227 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffda832-0b8b-4e0e-93a9-2cada0965d77 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:07.790507 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffda832-0b8b-4e0e-93a9-2cada0965d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:07.790820 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffda832-0b8b-4e0e-93a9-2cada0965d77 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:07.791452 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffda832-0b8b-4e0e-93a9-2cada0965d77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:33:07.791751 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffda832-0b8b-4e0e-93a9-2cada0965d77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:07.793348 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffda832-0b8b-4e0e-93a9-2cada0965d77 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:07.804886 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 942/1881] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:33:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:33:07.808880 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffda832-0b8b-4e0e-93a9-2cada0965d77 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:07.818445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffda832-0b8b-4e0e-93a9-2cada0965d77 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:07.823164 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82457565-739a-4c66-94d8-83eb1c8538b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:07.828011 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82457565-739a-4c66-94d8-83eb1c8538b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:07.828011 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82457565-739a-4c66-94d8-83eb1c8538b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:07.828011 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82457565-739a-4c66-94d8-83eb1c8538b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:07.828011 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82457565-739a-4c66-94d8-83eb1c8538b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:07.828011 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82457565-739a-4c66-94d8-83eb1c8538b0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:07.862580 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 940/1882] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:33:07 2022] POST /identity/v3/projects => generated 365 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:33:07.863554 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148dc57-138f-45b2-ae84-694af828c2fb admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:07.865279 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2148dc57-138f-45b2-ae84-694af828c2fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:07.937209 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2148dc57-138f-45b2-ae84-694af828c2fb admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:07.941384 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 943/1883] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:33:07 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:33:07.944288 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82457565-739a-4c66-94d8-83eb1c8538b0 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:08.010826 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09073f27-bd8f-4a31-84fb-6517e9a90d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:08.011447 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09073f27-bd8f-4a31-84fb-6517e9a90d32 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:08.011616 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09073f27-bd8f-4a31-84fb-6517e9a90d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:08.011883 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09073f27-bd8f-4a31-84fb-6517e9a90d32 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:08.012592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09073f27-bd8f-4a31-84fb-6517e9a90d32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:08.014532 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09073f27-bd8f-4a31-84fb-6517e9a90d32 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:08.018114 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82457565-739a-4c66-94d8-83eb1c8538b0 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:08.026107 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a40858-0b85-4d0e-9968-ecf399618fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:08.027010 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a40858-0b85-4d0e-9968-ecf399618fe9 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:08.033180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a40858-0b85-4d0e-9968-ecf399618fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:08.033180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a40858-0b85-4d0e-9968-ecf399618fe9 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:08.033180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a40858-0b85-4d0e-9968-ecf399618fe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:08.033180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a40858-0b85-4d0e-9968-ecf399618fe9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:08.112059 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 941/1884] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:33:07 2022] POST /identity/v3/users => generated 289 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:08.113273 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09073f27-bd8f-4a31-84fb-6517e9a90d32 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:08.123024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09073f27-bd8f-4a31-84fb-6517e9a90d32 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:08.127795 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd193a0-6273-4ede-8a3b-ff286b335ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:08.129436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd193a0-6273-4ede-8a3b-ff286b335ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:08.129436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd193a0-6273-4ede-8a3b-ff286b335ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:08.129655 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd193a0-6273-4ede-8a3b-ff286b335ae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:08.130449 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd193a0-6273-4ede-8a3b-ff286b335ae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:08.132074 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd193a0-6273-4ede-8a3b-ff286b335ae1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:08.143047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 942/1885] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:33:08.147900 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd193a0-6273-4ede-8a3b-ff286b335ae1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:08.156458 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 944/1886] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:33:07 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:08.164925 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a40858-0b85-4d0e-9968-ecf399618fe9 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:08.175554 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd193a0-6273-4ede-8a3b-ff286b335ae1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:08.181901 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85564056-edb6-43d5-9826-57827ff5ac2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:08.182922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85564056-edb6-43d5-9826-57827ff5ac2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:08.183208 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85564056-edb6-43d5-9826-57827ff5ac2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:08.183485 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85564056-edb6-43d5-9826-57827ff5ac2b admin admin] PATH_INFO: `/v3/projects/c332488907d34a00a90086b62617055b/users/0675d69ba9504312a253eb8c910d9593/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:08.184806 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a40858-0b85-4d0e-9968-ecf399618fe9 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:08.190050 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af34aaf-b686-4783-8e66-c27576c99f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:08.192675 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af34aaf-b686-4783-8e66-c27576c99f71 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:08.192675 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af34aaf-b686-4783-8e66-c27576c99f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:08.192675 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af34aaf-b686-4783-8e66-c27576c99f71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:08.192675 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af34aaf-b686-4783-8e66-c27576c99f71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:08.194533 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af34aaf-b686-4783-8e66-c27576c99f71 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:08.199231 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85564056-edb6-43d5-9826-57827ff5ac2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c332488907d34a00a90086b62617055b, user_id=0675d69ba9504312a253eb8c910d9593, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:08.201868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85564056-edb6-43d5-9826-57827ff5ac2b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:08.209512 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 945/1887] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:33:08.210628 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af34aaf-b686-4783-8e66-c27576c99f71 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:08.220482 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af34aaf-b686-4783-8e66-c27576c99f71 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:08.230058 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac29168-d532-4608-82bb-085cef0d1e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:08.230058 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac29168-d532-4608-82bb-085cef0d1e15 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:08.230058 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac29168-d532-4608-82bb-085cef0d1e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:08.230058 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac29168-d532-4608-82bb-085cef0d1e15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:08.230764 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac29168-d532-4608-82bb-085cef0d1e15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:08.234878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac29168-d532-4608-82bb-085cef0d1e15 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:08.246739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 946/1888] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:08.249747 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac29168-d532-4608-82bb-085cef0d1e15 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:08.260694 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac29168-d532-4608-82bb-085cef0d1e15 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:08.265232 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7756f18-49ec-429c-a902-eb7337090cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:08.265862 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7756f18-49ec-429c-a902-eb7337090cad admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:08.266143 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7756f18-49ec-429c-a902-eb7337090cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:08.266418 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7756f18-49ec-429c-a902-eb7337090cad admin admin] PATH_INFO: `/v3/projects/a9b1bba8886b4af593ae2887de2df3f5/users/fc0887a9dd884ab28f0e75aa46301e6c/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:08.280893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7756f18-49ec-429c-a902-eb7337090cad admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9b1bba8886b4af593ae2887de2df3f5, user_id=fc0887a9dd884ab28f0e75aa46301e6c, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:08.283550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7756f18-49ec-429c-a902-eb7337090cad admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:08.294998 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 943/1889] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:08 2022] PUT /identity/v3/projects/c332488907d34a00a90086b62617055b/users/0675d69ba9504312a253eb8c910d9593/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:33:08.298755 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85564056-edb6-43d5-9826-57827ff5ac2b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:08.360591 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85564056-edb6-43d5-9826-57827ff5ac2b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:08.365318 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e633d3-1c84-4739-9a47-176777ede38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:08.365968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e633d3-1c84-4739-9a47-176777ede38a admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:08.366250 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e633d3-1c84-4739-9a47-176777ede38a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:08.366519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e633d3-1c84-4739-9a47-176777ede38a admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:08.367205 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e633d3-1c84-4739-9a47-176777ede38a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:08.368796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e633d3-1c84-4739-9a47-176777ede38a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:08.378504 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 944/1890] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:33:08.383469 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e633d3-1c84-4739-9a47-176777ede38a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:08.393027 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e633d3-1c84-4739-9a47-176777ede38a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:08.399081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 947/1891] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:08 2022] PUT /identity/v3/projects/a9b1bba8886b4af593ae2887de2df3f5/users/fc0887a9dd884ab28f0e75aa46301e6c/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:33:08.410450 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05d50d-b7aa-46c1-aa42-6a100a6e0c2f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:08.410833 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05d50d-b7aa-46c1-aa42-6a100a6e0c2f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:08.411176 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e05d50d-b7aa-46c1-aa42-6a100a6e0c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:08.446961 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1e05d50d-b7aa-46c1-aa42-6a100a6e0c2f None None] MFA Rules not processed for user `fc0887a9dd884ab28f0e75aa46301e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:08.446961 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1e05d50d-b7aa-46c1-aa42-6a100a6e0c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:08.454206 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f066c3ce-169b-49b6-8b46-6d21a335b149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:08.454913 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066c3ce-169b-49b6-8b46-6d21a335b149 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:08.455313 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066c3ce-169b-49b6-8b46-6d21a335b149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:08.455592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066c3ce-169b-49b6-8b46-6d21a335b149 admin admin] PATH_INFO: `/v3/projects/c332488907d34a00a90086b62617055b/users/0675d69ba9504312a253eb8c910d9593/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:08.457581 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f066c3ce-169b-49b6-8b46-6d21a335b149 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c332488907d34a00a90086b62617055b, user_id=0675d69ba9504312a253eb8c910d9593, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:08.460227 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f066c3ce-169b-49b6-8b46-6d21a335b149 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:08.471222 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 945/1892] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:08 2022] PUT /identity/v3/projects/c332488907d34a00a90086b62617055b/users/0675d69ba9504312a253eb8c910d9593/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:33:08.487442 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776f0cc-a784-4f85-b943-4c095ec84e00 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:08.487738 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776f0cc-a784-4f85-b943-4c095ec84e00 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:08.488639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776f0cc-a784-4f85-b943-4c095ec84e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:08.527202 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0776f0cc-a784-4f85-b943-4c095ec84e00 None None] MFA Rules not processed for user `0675d69ba9504312a253eb8c910d9593`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:08.528413 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0776f0cc-a784-4f85-b943-4c095ec84e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:08.531551 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 948/1893] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:08.539433 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05d50d-b7aa-46c1-aa42-6a100a6e0c2f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:08.551263 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e05d50d-b7aa-46c1-aa42-6a100a6e0c2f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:08.615047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 946/1894] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:08 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:33:08.626913 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776f0cc-a784-4f85-b943-4c095ec84e00 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:08.631588 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0776f0cc-a784-4f85-b943-4c095ec84e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:08.631765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84820066-2ac0-4911-8568-31860f513aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:08.641771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84820066-2ac0-4911-8568-31860f513aa4 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:08.641771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84820066-2ac0-4911-8568-31860f513aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:08.641771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84820066-2ac0-4911-8568-31860f513aa4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:08.641771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84820066-2ac0-4911-8568-31860f513aa4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:08.641771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84820066-2ac0-4911-8568-31860f513aa4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:08.662331 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:08.690557 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776f0cc-a784-4f85-b943-4c095ec84e00 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:08.694291 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee8060a-daec-41dd-ac42-b35a6cc5088d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:08.694962 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8060a-daec-41dd-ac42-b35a6cc5088d service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:08.695246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8060a-daec-41dd-ac42-b35a6cc5088d service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:08.695541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee8060a-daec-41dd-ac42-b35a6cc5088d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:08.705221 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee8060a-daec-41dd-ac42-b35a6cc5088d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:08.707448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee8060a-daec-41dd-ac42-b35a6cc5088d service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:08.717756 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 947/1895] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:08 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:33:08.725182 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 949/1896] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:33:08 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:33:08.726213 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee8060a-daec-41dd-ac42-b35a6cc5088d service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:08.737284 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee8060a-daec-41dd-ac42-b35a6cc5088d service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:08.795858 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae088717-d8c2-4787-8bea-121651359924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:08.796474 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae088717-d8c2-4787-8bea-121651359924 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:08.796752 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae088717-d8c2-4787-8bea-121651359924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:08.797023 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae088717-d8c2-4787-8bea-121651359924 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:08.797668 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae088717-d8c2-4787-8bea-121651359924 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:08.799640 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae088717-d8c2-4787-8bea-121651359924 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:08.892609 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 948/1897] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:33:08 2022] POST /identity/v3/users => generated 299 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:33:08.897114 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84820066-2ac0-4911-8568-31860f513aa4 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:08.907437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84820066-2ac0-4911-8568-31860f513aa4 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:08.912210 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff801a0d-86df-4da2-b5ed-a468b62ddb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:08.912812 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff801a0d-86df-4da2-b5ed-a468b62ddb55 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:08.913142 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff801a0d-86df-4da2-b5ed-a468b62ddb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:08.913522 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff801a0d-86df-4da2-b5ed-a468b62ddb55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:08.914161 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff801a0d-86df-4da2-b5ed-a468b62ddb55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:08.915788 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff801a0d-86df-4da2-b5ed-a468b62ddb55 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:08.925665 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 950/1898] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:33:08.930856 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae088717-d8c2-4787-8bea-121651359924 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:08.939575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae088717-d8c2-4787-8bea-121651359924 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:08.944704 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0d124-3ed3-47b3-ab93-85a1e2bbbf47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:08.945358 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0d124-3ed3-47b3-ab93-85a1e2bbbf47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:08.945641 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0d124-3ed3-47b3-ab93-85a1e2bbbf47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:08.945917 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0d124-3ed3-47b3-ab93-85a1e2bbbf47 admin admin] PATH_INFO: `/v3/projects/355c5ae00f0049ad809df1d340689189/users/59f46bfbe85d4f00b3804dfdb4a07053/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:08.961594 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0d124-3ed3-47b3-ab93-85a1e2bbbf47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=355c5ae00f0049ad809df1d340689189, user_id=59f46bfbe85d4f00b3804dfdb4a07053, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:08.964293 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0d124-3ed3-47b3-ab93-85a1e2bbbf47 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:09.049697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 949/1899] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:08 2022] PUT /identity/v3/projects/355c5ae00f0049ad809df1d340689189/users/59f46bfbe85d4f00b3804dfdb4a07053/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:09.054173 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff801a0d-86df-4da2-b5ed-a468b62ddb55 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:09.064258 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff801a0d-86df-4da2-b5ed-a468b62ddb55 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:09.097589 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:09.124009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ca63a-146e-4dac-b364-558ff9e2b146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:09.124682 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ca63a-146e-4dac-b364-558ff9e2b146 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:09.126189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ca63a-146e-4dac-b364-558ff9e2b146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:09.126189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ca63a-146e-4dac-b364-558ff9e2b146 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:09.126189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ca63a-146e-4dac-b364-558ff9e2b146 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:09.127474 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ca63a-146e-4dac-b364-558ff9e2b146 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:09.138932 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 951/1900] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:09.142910 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0d124-3ed3-47b3-ab93-85a1e2bbbf47 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:09.154300 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0d124-3ed3-47b3-ab93-85a1e2bbbf47 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:09.159360 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee78d98-ca25-46d5-84f8-8067ad999503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:09.160151 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee78d98-ca25-46d5-84f8-8067ad999503 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:09.160454 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee78d98-ca25-46d5-84f8-8067ad999503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:09.160714 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee78d98-ca25-46d5-84f8-8067ad999503 admin admin] PATH_INFO: `/v3/projects/355c5ae00f0049ad809df1d340689189/users/59f46bfbe85d4f00b3804dfdb4a07053/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:09.162638 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee78d98-ca25-46d5-84f8-8067ad999503 admin admin] RBAC: Authorizing `identity:create_grant(project_id=355c5ae00f0049ad809df1d340689189, user_id=59f46bfbe85d4f00b3804dfdb4a07053, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:09.165371 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee78d98-ca25-46d5-84f8-8067ad999503 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:09.176970 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 950/1901] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:09 2022] PUT /identity/v3/projects/355c5ae00f0049ad809df1d340689189/users/59f46bfbe85d4f00b3804dfdb4a07053/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:09.188336 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01b24d-a552-4043-b832-cb342a209d62 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:09.191823 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01b24d-a552-4043-b832-cb342a209d62 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:09.191823 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01b24d-a552-4043-b832-cb342a209d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:09.218934 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ce01b24d-a552-4043-b832-cb342a209d62 None None] MFA Rules not processed for user `59f46bfbe85d4f00b3804dfdb4a07053`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:09.220126 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ce01b24d-a552-4043-b832-cb342a209d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:09.288773 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 952/1902] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:09 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:09.299556 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee78d98-ca25-46d5-84f8-8067ad999503 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:09.302494 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3ee78d98-ca25-46d5-84f8-8067ad999503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:09.361218 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee78d98-ca25-46d5-84f8-8067ad999503 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:09.365509 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45da0db6-5515-489a-830f-7febd7d0ea5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:09.366118 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da0db6-5515-489a-830f-7febd7d0ea5d service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:09.366399 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da0db6-5515-489a-830f-7febd7d0ea5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:09.366681 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45da0db6-5515-489a-830f-7febd7d0ea5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:09.376157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da0db6-5515-489a-830f-7febd7d0ea5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:09.377631 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45da0db6-5515-489a-830f-7febd7d0ea5d service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:09.388410 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 951/1903] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:33:09.789114 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce01b24d-a552-4043-b832-cb342a209d62 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:09.852959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce01b24d-a552-4043-b832-cb342a209d62 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:09.857847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb044b03-cfa5-4572-ac58-5f4f53cb9a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:09.858495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb044b03-cfa5-4572-ac58-5f4f53cb9a29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:09.858812 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb044b03-cfa5-4572-ac58-5f4f53cb9a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:09.859094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb044b03-cfa5-4572-ac58-5f4f53cb9a29 admin admin] PATH_INFO: `/v3/users/c463ee48d3d04c698a4c465f5453ddad` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:09.860245 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb044b03-cfa5-4572-ac58-5f4f53cb9a29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c463ee48d3d04c698a4c465f5453ddad)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:09.862176 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb044b03-cfa5-4572-ac58-5f4f53cb9a29 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:10.087080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-bb044b03-cfa5-4572-ac58-5f4f53cb9a29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c463ee48d3d04c698a4c465f5453ddad'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:10.094373 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-bb044b03-cfa5-4572-ac58-5f4f53cb9a29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c463ee48d3d04c698a4c465f5453ddad'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:10.168843 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-bb044b03-cfa5-4572-ac58-5f4f53cb9a29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c463ee48d3d04c698a4c465f5453ddad'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:10.169467 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-bb044b03-cfa5-4572-ac58-5f4f53cb9a29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c463ee48d3d04c698a4c465f5453ddad'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:10.181728 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-bb044b03-cfa5-4572-ac58-5f4f53cb9a29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c463ee48d3d04c698a4c465f5453ddad'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:10.187586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 953/1904] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:33:09 2022] DELETE /identity/v3/users/c463ee48d3d04c698a4c465f5453ddad => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:10.192133 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:10.192368 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b6a1c-5da2-4dbb-9cef-d41be033fb04 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:10.192808 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b6a1c-5da2-4dbb-9cef-d41be033fb04 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:10.192955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b6a1c-5da2-4dbb-9cef-d41be033fb04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:10.212686 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0b3b6a1c-5da2-4dbb-9cef-d41be033fb04 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:10.213983 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0b3b6a1c-5da2-4dbb-9cef-d41be033fb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:10.279659 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 952/1905] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:10.286813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb044b03-cfa5-4572-ac58-5f4f53cb9a29 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:10.288278 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bb044b03-cfa5-4572-ac58-5f4f53cb9a29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:10.344058 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb044b03-cfa5-4572-ac58-5f4f53cb9a29 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:10.347845 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5f472-7b33-4a64-871c-371eb346f80c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:10.348622 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5f472-7b33-4a64-871c-371eb346f80c service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:10.348943 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5f472-7b33-4a64-871c-371eb346f80c service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:10.349232 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a5f472-7b33-4a64-871c-371eb346f80c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:10.358782 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a5f472-7b33-4a64-871c-371eb346f80c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:10.360214 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a5f472-7b33-4a64-871c-371eb346f80c service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:10.369685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 954/1906] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:33:10.452889 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:10.564280 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b6a1c-5da2-4dbb-9cef-d41be033fb04 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:10.566096 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0b3b6a1c-5da2-4dbb-9cef-d41be033fb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:10.580798 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b6a1c-5da2-4dbb-9cef-d41be033fb04 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:10.586137 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8987c-2525-4610-b5a7-b32c43d35c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:10.586915 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8987c-2525-4610-b5a7-b32c43d35c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:10.587269 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8987c-2525-4610-b5a7-b32c43d35c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:10.588167 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8987c-2525-4610-b5a7-b32c43d35c1a admin admin] PATH_INFO: `/v3/users/fc0887a9dd884ab28f0e75aa46301e6c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:10.588839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a8987c-2525-4610-b5a7-b32c43d35c1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc0887a9dd884ab28f0e75aa46301e6c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:10.589600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:10.590954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a8987c-2525-4610-b5a7-b32c43d35c1a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:10.773259 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-72a8987c-2525-4610-b5a7-b32c43d35c1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc0887a9dd884ab28f0e75aa46301e6c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:10.788975 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5f472-7b33-4a64-871c-371eb346f80c service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:10.791965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f1a5f472-7b33-4a64-871c-371eb346f80c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:10.815032 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a5f472-7b33-4a64-871c-371eb346f80c service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:10.820783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdb608-d7c8-4457-a821-89ce296b2a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:10.821320 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdb608-d7c8-4457-a821-89ce296b2a3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:10.821600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdb608-d7c8-4457-a821-89ce296b2a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:10.821858 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcdb608-d7c8-4457-a821-89ce296b2a3a admin admin] PATH_INFO: `/v3/projects/a08212d9edd84d86929d313237a29697` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:10.823027 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcdb608-d7c8-4457-a821-89ce296b2a3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a08212d9edd84d86929d313237a29697)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:10.824961 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcdb608-d7c8-4457-a821-89ce296b2a3a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:10.855514 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-72a8987c-2525-4610-b5a7-b32c43d35c1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc0887a9dd884ab28f0e75aa46301e6c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:10.856490 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-72a8987c-2525-4610-b5a7-b32c43d35c1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc0887a9dd884ab28f0e75aa46301e6c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:10.857099 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-72a8987c-2525-4610-b5a7-b32c43d35c1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc0887a9dd884ab28f0e75aa46301e6c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:10.871765 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-72a8987c-2525-4610-b5a7-b32c43d35c1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc0887a9dd884ab28f0e75aa46301e6c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:10.886102 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 953/1907] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:33:10 2022] DELETE /identity/v3/users/fc0887a9dd884ab28f0e75aa46301e6c => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:10.890513 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193ca2e-31aa-4e30-b1ac-ee7f2234e00a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:10.890893 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193ca2e-31aa-4e30-b1ac-ee7f2234e00a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:10.891242 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193ca2e-31aa-4e30-b1ac-ee7f2234e00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:10.911036 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f193ca2e-31aa-4e30-b1ac-ee7f2234e00a None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:10.912237 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f193ca2e-31aa-4e30-b1ac-ee7f2234e00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:10.947134 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-efcdb608-d7c8-4457-a821-89ce296b2a3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a08212d9edd84d86929d313237a29697', 'request_id': 'req-efcdb608-d7c8-4457-a821-89ce296b2a3a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:10.970413 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 954/1908] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:33:10.980123 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193ca2e-31aa-4e30-b1ac-ee7f2234e00a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:10.981528 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f193ca2e-31aa-4e30-b1ac-ee7f2234e00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:11.039051 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193ca2e-31aa-4e30-b1ac-ee7f2234e00a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:11.043211 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-efcdb608-d7c8-4457-a821-89ce296b2a3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a08212d9edd84d86929d313237a29697', 'request_id': 'req-efcdb608-d7c8-4457-a821-89ce296b2a3a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:11.049137 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-efcdb608-d7c8-4457-a821-89ce296b2a3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a08212d9edd84d86929d313237a29697', 'request_id': 'req-efcdb608-d7c8-4457-a821-89ce296b2a3a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:11.060559 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-efcdb608-d7c8-4457-a821-89ce296b2a3a admin admin] The token cache is being invalidate because project a08212d9edd84d86929d313237a29697 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:33:11.060850 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-efcdb608-d7c8-4457-a821-89ce296b2a3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:11.061469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-efcdb608-d7c8-4457-a821-89ce296b2a3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:11.062987 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 955/1909] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:33:10 2022] DELETE /identity/v3/projects/a08212d9edd84d86929d313237a29697 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:33:11.066872 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdb608-d7c8-4457-a821-89ce296b2a3a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:11.068151 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-efcdb608-d7c8-4457-a821-89ce296b2a3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:11.088999 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab84999-c66f-4b41-884a-d03cfd231777 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:11.089559 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab84999-c66f-4b41-884a-d03cfd231777 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:11.089842 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab84999-c66f-4b41-884a-d03cfd231777 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:11.090115 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab84999-c66f-4b41-884a-d03cfd231777 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:11.092236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1ab84999-c66f-4b41-884a-d03cfd231777 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:11.092713 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:11.142882 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcdb608-d7c8-4457-a821-89ce296b2a3a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:11.148228 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f364b4-7aa8-4d19-8e59-4b93cf62f087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:11.149168 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f364b4-7aa8-4d19-8e59-4b93cf62f087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:11.149627 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f364b4-7aa8-4d19-8e59-4b93cf62f087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:11.150274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f364b4-7aa8-4d19-8e59-4b93cf62f087 admin admin] PATH_INFO: `/v3/users/0675d69ba9504312a253eb8c910d9593` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:11.152536 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f364b4-7aa8-4d19-8e59-4b93cf62f087 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0675d69ba9504312a253eb8c910d9593)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:11.155082 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f364b4-7aa8-4d19-8e59-4b93cf62f087 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:11.159306 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab84999-c66f-4b41-884a-d03cfd231777 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:11.161443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab84999-c66f-4b41-884a-d03cfd231777 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:11.172201 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 955/1910] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:33:11.392172 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d3f364b4-7aa8-4d19-8e59-4b93cf62f087 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0675d69ba9504312a253eb8c910d9593'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:11.401952 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d3f364b4-7aa8-4d19-8e59-4b93cf62f087 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0675d69ba9504312a253eb8c910d9593'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:11.523515 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d3f364b4-7aa8-4d19-8e59-4b93cf62f087 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0675d69ba9504312a253eb8c910d9593'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:11.524135 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d3f364b4-7aa8-4d19-8e59-4b93cf62f087 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0675d69ba9504312a253eb8c910d9593'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:11.536127 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d3f364b4-7aa8-4d19-8e59-4b93cf62f087 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0675d69ba9504312a253eb8c910d9593'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:11.541845 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 956/1911] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:33:11 2022] DELETE /identity/v3/users/0675d69ba9504312a253eb8c910d9593 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:11.672730 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab84999-c66f-4b41-884a-d03cfd231777 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:11.676624 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1ab84999-c66f-4b41-884a-d03cfd231777 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:11.708273 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab84999-c66f-4b41-884a-d03cfd231777 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:11.777958 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e201c-6ee6-48df-af01-c39b859519ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:11.778598 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e201c-6ee6-48df-af01-c39b859519ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:11.778913 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e201c-6ee6-48df-af01-c39b859519ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:11.779191 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e201c-6ee6-48df-af01-c39b859519ec admin admin] PATH_INFO: `/v3/projects/a9b1bba8886b4af593ae2887de2df3f5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:11.780303 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e201c-6ee6-48df-af01-c39b859519ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9b1bba8886b4af593ae2887de2df3f5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:11.782209 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e201c-6ee6-48df-af01-c39b859519ec admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:11.890062 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6d6e201c-6ee6-48df-af01-c39b859519ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9b1bba8886b4af593ae2887de2df3f5', 'request_id': 'req-6d6e201c-6ee6-48df-af01-c39b859519ec'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:11.897779 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6d6e201c-6ee6-48df-af01-c39b859519ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9b1bba8886b4af593ae2887de2df3f5', 'request_id': 'req-6d6e201c-6ee6-48df-af01-c39b859519ec'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:11.907059 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6d6e201c-6ee6-48df-af01-c39b859519ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9b1bba8886b4af593ae2887de2df3f5', 'request_id': 'req-6d6e201c-6ee6-48df-af01-c39b859519ec'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:12.071644 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6d6e201c-6ee6-48df-af01-c39b859519ec admin admin] The token cache is being invalidate because project a9b1bba8886b4af593ae2887de2df3f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:33:12.071935 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6d6e201c-6ee6-48df-af01-c39b859519ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:12.072611 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6d6e201c-6ee6-48df-af01-c39b859519ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:12.074109 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 956/1912] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:33:11 2022] DELETE /identity/v3/projects/a9b1bba8886b4af593ae2887de2df3f5 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:12.078139 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f364b4-7aa8-4d19-8e59-4b93cf62f087 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:12.079598 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d3f364b4-7aa8-4d19-8e59-4b93cf62f087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:12.092422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f364b4-7aa8-4d19-8e59-4b93cf62f087 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:12.131804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e201c-6ee6-48df-af01-c39b859519ec admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:12.133593 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6d6e201c-6ee6-48df-af01-c39b859519ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:12.152609 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea28003-d511-4913-95d3-0ec362d793dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:12.153123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea28003-d511-4913-95d3-0ec362d793dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:12.153405 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea28003-d511-4913-95d3-0ec362d793dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:12.153680 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea28003-d511-4913-95d3-0ec362d793dd admin admin] PATH_INFO: `/v3/users/59f46bfbe85d4f00b3804dfdb4a07053` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:12.154868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea28003-d511-4913-95d3-0ec362d793dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=59f46bfbe85d4f00b3804dfdb4a07053)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:12.157167 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea28003-d511-4913-95d3-0ec362d793dd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:12.203370 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e201c-6ee6-48df-af01-c39b859519ec admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:12.208553 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478a8fb-e788-48c5-912a-985325636ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:12.209186 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478a8fb-e788-48c5-912a-985325636ce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:12.209464 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478a8fb-e788-48c5-912a-985325636ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:12.209735 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478a8fb-e788-48c5-912a-985325636ce6 admin admin] PATH_INFO: `/v3/projects/c332488907d34a00a90086b62617055b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:12.210874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c478a8fb-e788-48c5-912a-985325636ce6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c332488907d34a00a90086b62617055b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:12.213133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c478a8fb-e788-48c5-912a-985325636ce6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:12.232097 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:12.422106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-eea28003-d511-4913-95d3-0ec362d793dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59f46bfbe85d4f00b3804dfdb4a07053'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:12.428625 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c478a8fb-e788-48c5-912a-985325636ce6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c332488907d34a00a90086b62617055b', 'request_id': 'req-c478a8fb-e788-48c5-912a-985325636ce6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:12.433203 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-eea28003-d511-4913-95d3-0ec362d793dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59f46bfbe85d4f00b3804dfdb4a07053'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:12.434945 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c478a8fb-e788-48c5-912a-985325636ce6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c332488907d34a00a90086b62617055b', 'request_id': 'req-c478a8fb-e788-48c5-912a-985325636ce6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:12.441622 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c478a8fb-e788-48c5-912a-985325636ce6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c332488907d34a00a90086b62617055b', 'request_id': 'req-c478a8fb-e788-48c5-912a-985325636ce6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:12.528956 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-eea28003-d511-4913-95d3-0ec362d793dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59f46bfbe85d4f00b3804dfdb4a07053'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:12.529579 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-eea28003-d511-4913-95d3-0ec362d793dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59f46bfbe85d4f00b3804dfdb4a07053'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:12.542495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-eea28003-d511-4913-95d3-0ec362d793dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59f46bfbe85d4f00b3804dfdb4a07053'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:12.548441 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 957/1913] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:33:12 2022] DELETE /identity/v3/users/59f46bfbe85d4f00b3804dfdb4a07053 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:12.642673 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c478a8fb-e788-48c5-912a-985325636ce6 admin admin] The token cache is being invalidate because project c332488907d34a00a90086b62617055b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:33:12.643029 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c478a8fb-e788-48c5-912a-985325636ce6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:12.643651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c478a8fb-e788-48c5-912a-985325636ce6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:12.645852 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 957/1914] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:33:12 2022] DELETE /identity/v3/projects/c332488907d34a00a90086b62617055b => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:33:12.650631 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4929c-f18c-4561-9252-8ddcc8e389f1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:12.650950 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4929c-f18c-4561-9252-8ddcc8e389f1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:12.651227 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4929c-f18c-4561-9252-8ddcc8e389f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:12.661193 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:12.675399 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ebb4929c-f18c-4561-9252-8ddcc8e389f1 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:12.676730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ebb4929c-f18c-4561-9252-8ddcc8e389f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:12.737093 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 958/1915] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:12.745831 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91da23d-dcad-43a3-b695-46d1445d9ff0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:12.746419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91da23d-dcad-43a3-b695-46d1445d9ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:12.746591 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91da23d-dcad-43a3-b695-46d1445d9ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:12.767394 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a91da23d-dcad-43a3-b695-46d1445d9ff0 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:12.768674 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a91da23d-dcad-43a3-b695-46d1445d9ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:12.778013 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 958/1916] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:33:12.782483 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4929c-f18c-4561-9252-8ddcc8e389f1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:12.792193 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4929c-f18c-4561-9252-8ddcc8e389f1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:12.797382 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c532bf-5955-46d9-affe-70a20bfb92de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:12.798142 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c532bf-5955-46d9-affe-70a20bfb92de admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:12.798515 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c532bf-5955-46d9-affe-70a20bfb92de admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:12.798843 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c532bf-5955-46d9-affe-70a20bfb92de admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:12.799826 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c532bf-5955-46d9-affe-70a20bfb92de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:33:12.803225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c532bf-5955-46d9-affe-70a20bfb92de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:12.803225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c532bf-5955-46d9-affe-70a20bfb92de admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:12.813656 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 959/1917] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:33:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:33:12.819673 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91da23d-dcad-43a3-b695-46d1445d9ff0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:12.835107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91da23d-dcad-43a3-b695-46d1445d9ff0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:12.840228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb5d33-6fef-46d6-b40e-71472a287ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:12.841108 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb5d33-6fef-46d6-b40e-71472a287ab4 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:12.841492 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb5d33-6fef-46d6-b40e-71472a287ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:12.841986 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb5d33-6fef-46d6-b40e-71472a287ab4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:12.842685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb5d33-6fef-46d6-b40e-71472a287ab4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:12.845047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb5d33-6fef-46d6-b40e-71472a287ab4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:12.952057 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 959/1918] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:33:12 2022] POST /identity/v3/projects => generated 367 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:12.956548 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c532bf-5955-46d9-affe-70a20bfb92de admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:12.966550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c532bf-5955-46d9-affe-70a20bfb92de admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:13.026874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb5d33-6fef-46d6-b40e-71472a287ab4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:13.028490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823d407-a696-4d0b-abc2-5aee204880c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:13.029106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823d407-a696-4d0b-abc2-5aee204880c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:13.029397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823d407-a696-4d0b-abc2-5aee204880c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:13.029721 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823d407-a696-4d0b-abc2-5aee204880c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:13.030467 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823d407-a696-4d0b-abc2-5aee204880c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:13.032397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823d407-a696-4d0b-abc2-5aee204880c5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:13.035688 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-23eb5d33-6fef-46d6-b40e-71472a287ab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:13.057385 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb5d33-6fef-46d6-b40e-71472a287ab4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:13.063042 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d091b-22f5-495d-a878-b0240eb16db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:13.064354 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d091b-22f5-495d-a878-b0240eb16db4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:13.065304 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d091b-22f5-495d-a878-b0240eb16db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:13.067272 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d091b-22f5-495d-a878-b0240eb16db4 admin admin] PATH_INFO: `/v3/projects/355c5ae00f0049ad809df1d340689189` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:13.069314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d091b-22f5-495d-a878-b0240eb16db4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=355c5ae00f0049ad809df1d340689189)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:13.071455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d091b-22f5-495d-a878-b0240eb16db4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:13.126753 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 960/1919] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:33:12 2022] POST /identity/v3/users => generated 284 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:13.131304 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823d407-a696-4d0b-abc2-5aee204880c5 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:13.141716 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823d407-a696-4d0b-abc2-5aee204880c5 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:13.148608 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db677d3f-ff33-4f55-be3f-817821dff8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:13.149513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db677d3f-ff33-4f55-be3f-817821dff8ae admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:13.151748 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db677d3f-ff33-4f55-be3f-817821dff8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:13.151748 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db677d3f-ff33-4f55-be3f-817821dff8ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:13.151748 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db677d3f-ff33-4f55-be3f-817821dff8ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:13.152605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db677d3f-ff33-4f55-be3f-817821dff8ae admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:13.163966 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 961/1920] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:33:13.167764 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db677d3f-ff33-4f55-be3f-817821dff8ae admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:13.178904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db677d3f-ff33-4f55-be3f-817821dff8ae admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:13.185490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46d1dd-d455-4175-bfac-9d07e4b7dbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:13.186112 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46d1dd-d455-4175-bfac-9d07e4b7dbce admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:13.186395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46d1dd-d455-4175-bfac-9d07e4b7dbce admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:13.186666 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46d1dd-d455-4175-bfac-9d07e4b7dbce admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:13.187503 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46d1dd-d455-4175-bfac-9d07e4b7dbce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:13.189239 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46d1dd-d455-4175-bfac-9d07e4b7dbce admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:13.201069 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 962/1921] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:33:13.206597 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46d1dd-d455-4175-bfac-9d07e4b7dbce admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:13.216417 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46d1dd-d455-4175-bfac-9d07e4b7dbce admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:13.221106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbbfbe6-d914-4d61-8d21-05857055b7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:13.221767 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbfbe6-d914-4d61-8d21-05857055b7e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:13.222046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbfbe6-d914-4d61-8d21-05857055b7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:13.222415 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbfbe6-d914-4d61-8d21-05857055b7e8 admin admin] PATH_INFO: `/v3/projects/2ae4e6db7fca447486b41890fb31c367/users/1d6f49522ad24af1affd4d25f9d3e8b1/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:13.238654 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbbfbe6-d914-4d61-8d21-05857055b7e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ae4e6db7fca447486b41890fb31c367, user_id=1d6f49522ad24af1affd4d25f9d3e8b1, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:13.243719 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbbfbe6-d914-4d61-8d21-05857055b7e8 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:13.250440 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9d3d091b-22f5-495d-a878-b0240eb16db4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '355c5ae00f0049ad809df1d340689189', 'request_id': 'req-9d3d091b-22f5-495d-a878-b0240eb16db4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:13.256732 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9d3d091b-22f5-495d-a878-b0240eb16db4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '355c5ae00f0049ad809df1d340689189', 'request_id': 'req-9d3d091b-22f5-495d-a878-b0240eb16db4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:13.262689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9d3d091b-22f5-495d-a878-b0240eb16db4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '355c5ae00f0049ad809df1d340689189', 'request_id': 'req-9d3d091b-22f5-495d-a878-b0240eb16db4'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:13.332042 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 963/1922] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:13 2022] PUT /identity/v3/projects/2ae4e6db7fca447486b41890fb31c367/users/1d6f49522ad24af1affd4d25f9d3e8b1/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:13.342636 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1eb8c7-ef2b-47ad-b9f6-18b36082a40e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:13.342952 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1eb8c7-ef2b-47ad-b9f6-18b36082a40e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:13.343222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1eb8c7-ef2b-47ad-b9f6-18b36082a40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:13.371910 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-cc1eb8c7-ef2b-47ad-b9f6-18b36082a40e None None] MFA Rules not processed for user `1d6f49522ad24af1affd4d25f9d3e8b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:13.373104 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cc1eb8c7-ef2b-47ad-b9f6-18b36082a40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:13.389121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9d3d091b-22f5-495d-a878-b0240eb16db4 admin admin] The token cache is being invalidate because project 355c5ae00f0049ad809df1d340689189 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:33:13.389467 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9d3d091b-22f5-495d-a878-b0240eb16db4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:13.392306 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-9d3d091b-22f5-495d-a878-b0240eb16db4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:13.393696 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 960/1923] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:33:13 2022] DELETE /identity/v3/projects/355c5ae00f0049ad809df1d340689189 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:33:13.397991 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfdff3-0a31-4b6c-97e4-f78fcbe8fcf9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:13.398278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfdff3-0a31-4b6c-97e4-f78fcbe8fcf9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:13.398550 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cfdff3-0a31-4b6c-97e4-f78fcbe8fcf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:13.423728 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f4cfdff3-0a31-4b6c-97e4-f78fcbe8fcf9 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:13.423728 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f4cfdff3-0a31-4b6c-97e4-f78fcbe8fcf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:13.446321 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 964/1924] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:13.453121 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1eb8c7-ef2b-47ad-b9f6-18b36082a40e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:13.454673 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cc1eb8c7-ef2b-47ad-b9f6-18b36082a40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:13.501795 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 961/1925] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:13.513735 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a491303-b9ba-4f0e-a2b1-d310636cb90f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:13.514119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a491303-b9ba-4f0e-a2b1-d310636cb90f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:13.514303 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a491303-b9ba-4f0e-a2b1-d310636cb90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:13.540829 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6a491303-b9ba-4f0e-a2b1-d310636cb90f None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:13.542599 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6a491303-b9ba-4f0e-a2b1-d310636cb90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:13.546366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1eb8c7-ef2b-47ad-b9f6-18b36082a40e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:13.554369 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9becf-32f6-42d6-bb74-7e3ad6b205e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:13.555035 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9becf-32f6-42d6-bb74-7e3ad6b205e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:13.555300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9becf-32f6-42d6-bb74-7e3ad6b205e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:13.555592 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9becf-32f6-42d6-bb74-7e3ad6b205e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:13.556233 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9becf-32f6-42d6-bb74-7e3ad6b205e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:13.558277 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9becf-32f6-42d6-bb74-7e3ad6b205e4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:13.560487 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 962/1926] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:33:13.570282 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a491303-b9ba-4f0e-a2b1-d310636cb90f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:13.581711 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a491303-b9ba-4f0e-a2b1-d310636cb90f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:13.587554 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985c496-ee2e-40f4-9718-b2fc3d64bcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:13.587821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985c496-ee2e-40f4-9718-b2fc3d64bcb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:13.587998 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985c496-ee2e-40f4-9718-b2fc3d64bcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:13.588270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985c496-ee2e-40f4-9718-b2fc3d64bcb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:13.588975 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f985c496-ee2e-40f4-9718-b2fc3d64bcb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:33:13.589270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f985c496-ee2e-40f4-9718-b2fc3d64bcb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:13.590890 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f985c496-ee2e-40f4-9718-b2fc3d64bcb1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:13.602943 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 963/1927] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:33:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:33:13.607057 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985c496-ee2e-40f4-9718-b2fc3d64bcb1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:13.616519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985c496-ee2e-40f4-9718-b2fc3d64bcb1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:13.621261 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8c836-cc84-4292-b166-99d070ac066e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:13.621768 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8c836-cc84-4292-b166-99d070ac066e admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:13.624033 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8c836-cc84-4292-b166-99d070ac066e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:13.624033 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b8c836-cc84-4292-b166-99d070ac066e admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:13.624033 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b8c836-cc84-4292-b166-99d070ac066e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:13.624854 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b8c836-cc84-4292-b166-99d070ac066e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:13.719657 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 964/1928] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:33:13 2022] POST /identity/v3/projects => generated 369 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:13.724794 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 965/1929] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:33:13 2022] POST /identity/v3/projects => generated 367 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:33:13.724924 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8c836-cc84-4292-b166-99d070ac066e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:13.725711 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9becf-32f6-42d6-bb74-7e3ad6b205e4 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:13.787850 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9becf-32f6-42d6-bb74-7e3ad6b205e4 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:13.791524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b8c836-cc84-4292-b166-99d070ac066e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:13.793054 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb5b8e-7fa3-4641-b26f-12a082952736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:13.799814 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb5b8e-7fa3-4641-b26f-12a082952736 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:13.799814 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb5b8e-7fa3-4641-b26f-12a082952736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:13.799814 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb5b8e-7fa3-4641-b26f-12a082952736 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:13.799814 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bb5b8e-7fa3-4641-b26f-12a082952736 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:13.800152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be678766-fc43-40e8-8d92-3c954606aab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:13.800245 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bb5b8e-7fa3-4641-b26f-12a082952736 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:13.800316 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be678766-fc43-40e8-8d92-3c954606aab0 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:13.800316 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be678766-fc43-40e8-8d92-3c954606aab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:13.800316 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be678766-fc43-40e8-8d92-3c954606aab0 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:13.800316 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be678766-fc43-40e8-8d92-3c954606aab0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:13.800316 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be678766-fc43-40e8-8d92-3c954606aab0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:13.872196 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 966/1930] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:33:13 2022] POST /identity/v3/users => generated 290 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:33:13.874630 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:13.876250 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb5b8e-7fa3-4641-b26f-12a082952736 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:13.884018 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 965/1931] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:33:13 2022] POST /identity/v3/users => generated 285 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:33:13.887933 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb5b8e-7fa3-4641-b26f-12a082952736 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:13.888681 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be678766-fc43-40e8-8d92-3c954606aab0 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:13.892501 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a828166-7fba-46e0-ad5b-b4e9ba1f0063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:13.893097 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a828166-7fba-46e0-ad5b-b4e9ba1f0063 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:13.893475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a828166-7fba-46e0-ad5b-b4e9ba1f0063 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:13.893639 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a828166-7fba-46e0-ad5b-b4e9ba1f0063 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:13.894278 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a828166-7fba-46e0-ad5b-b4e9ba1f0063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:13.895887 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a828166-7fba-46e0-ad5b-b4e9ba1f0063 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:13.898517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be678766-fc43-40e8-8d92-3c954606aab0 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:13.903317 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922976e-2904-4895-8a62-dcfc68e2f432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:13.903922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922976e-2904-4895-8a62-dcfc68e2f432 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:13.904201 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922976e-2904-4895-8a62-dcfc68e2f432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:13.904479 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922976e-2904-4895-8a62-dcfc68e2f432 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:13.905118 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922976e-2904-4895-8a62-dcfc68e2f432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:13.906659 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 967/1932] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:33:13.906796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922976e-2904-4895-8a62-dcfc68e2f432 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:13.914128 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a828166-7fba-46e0-ad5b-b4e9ba1f0063 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:13.919666 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 966/1933] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:33:13.920706 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922976e-2904-4895-8a62-dcfc68e2f432 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:13.924020 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a828166-7fba-46e0-ad5b-b4e9ba1f0063 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:13.929280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0fe1d-b3f2-4c4d-a6f8-c4b8123eff69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:13.929919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0fe1d-b3f2-4c4d-a6f8-c4b8123eff69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:13.930200 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0fe1d-b3f2-4c4d-a6f8-c4b8123eff69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:13.930474 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e0fe1d-b3f2-4c4d-a6f8-c4b8123eff69 admin admin] PATH_INFO: `/v3/projects/a1d012bab8e24e848d7a62a92828348d/users/cf5bbd02dc6d4c18a78cf245cc30071c/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:13.930746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922976e-2904-4895-8a62-dcfc68e2f432 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:13.935304 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37230168-e6f2-4bfc-8895-d9790758db2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:13.937630 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37230168-e6f2-4bfc-8895-d9790758db2f admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:13.937630 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37230168-e6f2-4bfc-8895-d9790758db2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:13.937630 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37230168-e6f2-4bfc-8895-d9790758db2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:13.937630 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37230168-e6f2-4bfc-8895-d9790758db2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:13.938794 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37230168-e6f2-4bfc-8895-d9790758db2f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:13.945058 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e0fe1d-b3f2-4c4d-a6f8-c4b8123eff69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1d012bab8e24e848d7a62a92828348d, user_id=cf5bbd02dc6d4c18a78cf245cc30071c, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:13.947744 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e0fe1d-b3f2-4c4d-a6f8-c4b8123eff69 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:13.950206 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 967/1934] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:33:13.954394 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37230168-e6f2-4bfc-8895-d9790758db2f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:13.963971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37230168-e6f2-4bfc-8895-d9790758db2f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:13.968616 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175c032-13e8-4b7b-ad94-2124421c7c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:13.969387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175c032-13e8-4b7b-ad94-2124421c7c42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:13.969663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175c032-13e8-4b7b-ad94-2124421c7c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:13.969935 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175c032-13e8-4b7b-ad94-2124421c7c42 admin admin] PATH_INFO: `/v3/projects/5e3423cc79d04e83a7e0e292ac973188/users/445190e0a56e4b409de1333d9e915019/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:13.984082 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0175c032-13e8-4b7b-ad94-2124421c7c42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e3423cc79d04e83a7e0e292ac973188, user_id=445190e0a56e4b409de1333d9e915019, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:13.986870 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0175c032-13e8-4b7b-ad94-2124421c7c42 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:14.011920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 968/1935] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:13 2022] PUT /identity/v3/projects/a1d012bab8e24e848d7a62a92828348d/users/cf5bbd02dc6d4c18a78cf245cc30071c/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:33:14.015863 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0fe1d-b3f2-4c4d-a6f8-c4b8123eff69 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:14.065384 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 968/1936] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:13 2022] PUT /identity/v3/projects/5e3423cc79d04e83a7e0e292ac973188/users/445190e0a56e4b409de1333d9e915019/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:33:14.071494 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ded89b-d787-478d-9ccf-e7bb90bb88a9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:14.071782 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ded89b-d787-478d-9ccf-e7bb90bb88a9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:14.072055 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ded89b-d787-478d-9ccf-e7bb90bb88a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:14.078636 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e0fe1d-b3f2-4c4d-a6f8-c4b8123eff69 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:14.094548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-79ded89b-d787-478d-9ccf-e7bb90bb88a9 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:14.095805 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-79ded89b-d787-478d-9ccf-e7bb90bb88a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:14.140494 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657e696-3620-4919-9699-0514d28781b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:14.141208 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657e696-3620-4919-9699-0514d28781b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:14.141486 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657e696-3620-4919-9699-0514d28781b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:14.141759 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c657e696-3620-4919-9699-0514d28781b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:14.142457 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c657e696-3620-4919-9699-0514d28781b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:14.144075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c657e696-3620-4919-9699-0514d28781b2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:14.154546 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 969/1937] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:14 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:33:14.159076 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657e696-3620-4919-9699-0514d28781b2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:14.162120 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:14.166754 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 969/1938] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:33:14.177376 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c657e696-3620-4919-9699-0514d28781b2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:14.177834 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ded89b-d787-478d-9ccf-e7bb90bb88a9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:14.179323 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-79ded89b-d787-478d-9ccf-e7bb90bb88a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:14.182236 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f4bce-d992-4c0c-b9f1-d39ad5ad87d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:14.182937 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f4bce-d992-4c0c-b9f1-d39ad5ad87d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:14.183220 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f4bce-d992-4c0c-b9f1-d39ad5ad87d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:14.183495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f4bce-d992-4c0c-b9f1-d39ad5ad87d1 admin admin] PATH_INFO: `/v3/projects/a1d012bab8e24e848d7a62a92828348d/users/cf5bbd02dc6d4c18a78cf245cc30071c/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:14.185402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f4bce-d992-4c0c-b9f1-d39ad5ad87d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1d012bab8e24e848d7a62a92828348d, user_id=cf5bbd02dc6d4c18a78cf245cc30071c, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:14.188126 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f4bce-d992-4c0c-b9f1-d39ad5ad87d1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:14.199099 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 970/1939] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:14 2022] PUT /identity/v3/projects/a1d012bab8e24e848d7a62a92828348d/users/cf5bbd02dc6d4c18a78cf245cc30071c/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:14.209106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c884b3-2913-443c-b16c-d80c449fd5f1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:14.209392 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c884b3-2913-443c-b16c-d80c449fd5f1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:14.209667 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c884b3-2913-443c-b16c-d80c449fd5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:14.237852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c5c884b3-2913-443c-b16c-d80c449fd5f1 None None] MFA Rules not processed for user `cf5bbd02dc6d4c18a78cf245cc30071c`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:14.239782 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c5c884b3-2913-443c-b16c-d80c449fd5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:14.252087 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ded89b-d787-478d-9ccf-e7bb90bb88a9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:14.304307 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004d22a4-ea14-4b97-8865-084306d46c44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:14.305005 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d22a4-ea14-4b97-8865-084306d46c44 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:14.305291 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d22a4-ea14-4b97-8865-084306d46c44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:14.305566 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d22a4-ea14-4b97-8865-084306d46c44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:14.323922 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 971/1940] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:14 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:33:14.334778 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c884b3-2913-443c-b16c-d80c449fd5f1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:14.336201 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c5c884b3-2913-443c-b16c-d80c449fd5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:14.374108 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004d22a4-ea14-4b97-8865-084306d46c44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:14.376710 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004d22a4-ea14-4b97-8865-084306d46c44 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:14.388717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 970/1941] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:33:14.398416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c884b3-2913-443c-b16c-d80c449fd5f1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:14.402413 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5885edbd-732c-47f3-bb57-f7b30b214e05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:14.403064 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885edbd-732c-47f3-bb57-f7b30b214e05 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:14.403343 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885edbd-732c-47f3-bb57-f7b30b214e05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:14.403607 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5885edbd-732c-47f3-bb57-f7b30b214e05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:14.413614 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5885edbd-732c-47f3-bb57-f7b30b214e05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:14.415070 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5885edbd-732c-47f3-bb57-f7b30b214e05 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:14.424859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 972/1942] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:14 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:33:15.114615 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:15.557563 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:15.911987 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077b7c0-b495-4de9-a7a0-0b58aab68ab8 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:15.912273 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077b7c0-b495-4de9-a7a0-0b58aab68ab8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:15.912544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077b7c0-b495-4de9-a7a0-0b58aab68ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:15.951706 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e077b7c0-b495-4de9-a7a0-0b58aab68ab8 None None] MFA Rules not processed for user `445190e0a56e4b409de1333d9e915019`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:15.953080 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e077b7c0-b495-4de9-a7a0-0b58aab68ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:16.046776 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 971/1943] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:16.054361 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5885edbd-732c-47f3-bb57-f7b30b214e05 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:16.069288 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5885edbd-732c-47f3-bb57-f7b30b214e05 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:16.076272 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52481108-d474-4742-962e-631f93af0ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:16.076885 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52481108-d474-4742-962e-631f93af0ac5 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:16.077161 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52481108-d474-4742-962e-631f93af0ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:16.077430 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52481108-d474-4742-962e-631f93af0ac5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:16.079987 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52481108-d474-4742-962e-631f93af0ac5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:16.090730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52481108-d474-4742-962e-631f93af0ac5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:16.186878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 973/1944] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:33:16 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:33:16.188427 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077b7c0-b495-4de9-a7a0-0b58aab68ab8 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:16.205228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077b7c0-b495-4de9-a7a0-0b58aab68ab8 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:16.299431 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f3a9c0-ac73-49c5-9763-2e635656e653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:16.300116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3a9c0-ac73-49c5-9763-2e635656e653 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:16.301030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3a9c0-ac73-49c5-9763-2e635656e653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:16.301452 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f3a9c0-ac73-49c5-9763-2e635656e653 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:16.302140 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f3a9c0-ac73-49c5-9763-2e635656e653 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:16.304129 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f3a9c0-ac73-49c5-9763-2e635656e653 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:16.389701 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 972/1945] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:33:16 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:16.390982 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52481108-d474-4742-962e-631f93af0ac5 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:16.409109 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52481108-d474-4742-962e-631f93af0ac5 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:16.417348 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f75b6-d71e-4589-b7c8-62efc33f0f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:16.418175 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f75b6-d71e-4589-b7c8-62efc33f0f00 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:16.418453 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f75b6-d71e-4589-b7c8-62efc33f0f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:16.418764 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f75b6-d71e-4589-b7c8-62efc33f0f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:16.419403 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f75b6-d71e-4589-b7c8-62efc33f0f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:16.420991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f75b6-d71e-4589-b7c8-62efc33f0f00 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:16.432135 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 974/1946] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:16.436467 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f3a9c0-ac73-49c5-9763-2e635656e653 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:16.446853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f3a9c0-ac73-49c5-9763-2e635656e653 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:16.451653 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ae44a-79f2-4cab-9309-cb90c819ec82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:16.452371 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ae44a-79f2-4cab-9309-cb90c819ec82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:16.452944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ae44a-79f2-4cab-9309-cb90c819ec82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:16.453156 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ae44a-79f2-4cab-9309-cb90c819ec82 admin admin] PATH_INFO: `/v3/projects/cd7bf0a45e214c0f93ad66e1f8dc68ec/users/fc54203f08f34a4dabf2ff1aeef8490c/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:16.468870 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ae44a-79f2-4cab-9309-cb90c819ec82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd7bf0a45e214c0f93ad66e1f8dc68ec, user_id=fc54203f08f34a4dabf2ff1aeef8490c, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:16.471590 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ae44a-79f2-4cab-9309-cb90c819ec82 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:16.559950 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 973/1947] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:16 2022] PUT /identity/v3/projects/cd7bf0a45e214c0f93ad66e1f8dc68ec/users/fc54203f08f34a4dabf2ff1aeef8490c/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:16.563812 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f75b6-d71e-4589-b7c8-62efc33f0f00 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:16.574068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f75b6-d71e-4589-b7c8-62efc33f0f00 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:16.642514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee7e39-ad57-443b-b5c4-422c317935b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:16.643162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee7e39-ad57-443b-b5c4-422c317935b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:16.643490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee7e39-ad57-443b-b5c4-422c317935b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:16.643762 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eee7e39-ad57-443b-b5c4-422c317935b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:16.644941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee7e39-ad57-443b-b5c4-422c317935b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:16.646502 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eee7e39-ad57-443b-b5c4-422c317935b3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:16.656233 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 975/1948] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:16.661114 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ae44a-79f2-4cab-9309-cb90c819ec82 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:16.672826 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ae44a-79f2-4cab-9309-cb90c819ec82 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:16.677811 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6531cb0-92fe-45d8-b615-85ce1a6b173e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:16.678588 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6531cb0-92fe-45d8-b615-85ce1a6b173e admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:16.678895 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6531cb0-92fe-45d8-b615-85ce1a6b173e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:16.679323 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6531cb0-92fe-45d8-b615-85ce1a6b173e admin admin] PATH_INFO: `/v3/projects/cd7bf0a45e214c0f93ad66e1f8dc68ec/users/fc54203f08f34a4dabf2ff1aeef8490c/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:16.681380 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6531cb0-92fe-45d8-b615-85ce1a6b173e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd7bf0a45e214c0f93ad66e1f8dc68ec, user_id=fc54203f08f34a4dabf2ff1aeef8490c, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:16.684177 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6531cb0-92fe-45d8-b615-85ce1a6b173e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:16.700925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 974/1949] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:16 2022] PUT /identity/v3/projects/cd7bf0a45e214c0f93ad66e1f8dc68ec/users/fc54203f08f34a4dabf2ff1aeef8490c/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:16.748546 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:16.903924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:17.569731 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee7e39-ad57-443b-b5c4-422c317935b3 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:17.581275 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eee7e39-ad57-443b-b5c4-422c317935b3 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:17.638074 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eadedf4-66d1-4fb6-91d1-05b81262e2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:17.638732 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadedf4-66d1-4fb6-91d1-05b81262e2ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:17.639017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadedf4-66d1-4fb6-91d1-05b81262e2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:17.639289 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eadedf4-66d1-4fb6-91d1-05b81262e2ff admin admin] PATH_INFO: `/v3/users/1d6f49522ad24af1affd4d25f9d3e8b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:17.640432 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eadedf4-66d1-4fb6-91d1-05b81262e2ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d6f49522ad24af1affd4d25f9d3e8b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:17.642332 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eadedf4-66d1-4fb6-91d1-05b81262e2ff admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:17.891488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2eadedf4-66d1-4fb6-91d1-05b81262e2ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d6f49522ad24af1affd4d25f9d3e8b1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:17.900107 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2eadedf4-66d1-4fb6-91d1-05b81262e2ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d6f49522ad24af1affd4d25f9d3e8b1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:17.997078 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2eadedf4-66d1-4fb6-91d1-05b81262e2ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d6f49522ad24af1affd4d25f9d3e8b1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:17.998046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2eadedf4-66d1-4fb6-91d1-05b81262e2ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d6f49522ad24af1affd4d25f9d3e8b1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:18.016838 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2eadedf4-66d1-4fb6-91d1-05b81262e2ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d6f49522ad24af1affd4d25f9d3e8b1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:18.028476 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 976/1950] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:33:17 2022] DELETE /identity/v3/users/1d6f49522ad24af1affd4d25f9d3e8b1 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:18.037505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d57f5-c57f-451c-a310-e4a061fb9007 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:18.037975 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d57f5-c57f-451c-a310-e4a061fb9007 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:18.038065 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d57f5-c57f-451c-a310-e4a061fb9007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:18.057800 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-521d57f5-c57f-451c-a310-e4a061fb9007 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:18.059068 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-521d57f5-c57f-451c-a310-e4a061fb9007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:18.099442 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9022e-11f9-4df8-a261-1749d4328e75 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:18.099915 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9022e-11f9-4df8-a261-1749d4328e75 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:18.100341 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9022e-11f9-4df8-a261-1749d4328e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:18.130253 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-dcc9022e-11f9-4df8-a261-1749d4328e75 None None] MFA Rules not processed for user `fc54203f08f34a4dabf2ff1aeef8490c`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:18.133290 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-dcc9022e-11f9-4df8-a261-1749d4328e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:18.133397 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 975/1951] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:33:18.142226 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d57f5-c57f-451c-a310-e4a061fb9007 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:18.143816 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-521d57f5-c57f-451c-a310-e4a061fb9007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:18.207394 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d57f5-c57f-451c-a310-e4a061fb9007 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:18.208911 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 977/1952] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:18 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:18.213897 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:18.216380 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a96f4-364b-4899-b616-f7f813500de3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:18.217079 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a96f4-364b-4899-b616-f7f813500de3 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:18.217431 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a96f4-364b-4899-b616-f7f813500de3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:18.217770 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a96f4-364b-4899-b616-f7f813500de3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:18.221001 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9022e-11f9-4df8-a261-1749d4328e75 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:18.222426 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-dcc9022e-11f9-4df8-a261-1749d4328e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:18.227967 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0a96f4-364b-4899-b616-f7f813500de3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:18.229395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0a96f4-364b-4899-b616-f7f813500de3 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:18.239310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 976/1953] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:33:18.281719 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9022e-11f9-4df8-a261-1749d4328e75 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:18.285461 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e0517-69ca-45e1-b11a-016742bf40e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:18.286090 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e0517-69ca-45e1-b11a-016742bf40e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:18.286376 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e0517-69ca-45e1-b11a-016742bf40e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:18.286687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e0517-69ca-45e1-b11a-016742bf40e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:18.288591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-602e0517-69ca-45e1-b11a-016742bf40e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:18.366014 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e0517-69ca-45e1-b11a-016742bf40e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:18.369436 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e0517-69ca-45e1-b11a-016742bf40e1 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:18.381472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 978/1954] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:33:18.603674 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:18.760440 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a96f4-364b-4899-b616-f7f813500de3 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:18.761634 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5c0a96f4-364b-4899-b616-f7f813500de3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:18.783808 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a96f4-364b-4899-b616-f7f813500de3 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:18.793125 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8628d116-766c-48fe-98f8-cb7f534bc60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:18.793980 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628d116-766c-48fe-98f8-cb7f534bc60e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:18.794323 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628d116-766c-48fe-98f8-cb7f534bc60e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:18.794605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8628d116-766c-48fe-98f8-cb7f534bc60e admin admin] PATH_INFO: `/v3/projects/2ae4e6db7fca447486b41890fb31c367` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:18.805277 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8628d116-766c-48fe-98f8-cb7f534bc60e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ae4e6db7fca447486b41890fb31c367)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:18.805277 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8628d116-766c-48fe-98f8-cb7f534bc60e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:18.939344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8628d116-766c-48fe-98f8-cb7f534bc60e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ae4e6db7fca447486b41890fb31c367', 'request_id': 'req-8628d116-766c-48fe-98f8-cb7f534bc60e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:18.946395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8628d116-766c-48fe-98f8-cb7f534bc60e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ae4e6db7fca447486b41890fb31c367', 'request_id': 'req-8628d116-766c-48fe-98f8-cb7f534bc60e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:18.955432 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8628d116-766c-48fe-98f8-cb7f534bc60e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ae4e6db7fca447486b41890fb31c367', 'request_id': 'req-8628d116-766c-48fe-98f8-cb7f534bc60e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:19.028255 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8628d116-766c-48fe-98f8-cb7f534bc60e admin admin] The token cache is being invalidate because project 2ae4e6db7fca447486b41890fb31c367 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:33:19.028503 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8628d116-766c-48fe-98f8-cb7f534bc60e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:19.029177 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8628d116-766c-48fe-98f8-cb7f534bc60e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:19.030879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 977/1955] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:33:18 2022] DELETE /identity/v3/projects/2ae4e6db7fca447486b41890fb31c367 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:19.035606 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e0517-69ca-45e1-b11a-016742bf40e1 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:19.052897 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e0517-69ca-45e1-b11a-016742bf40e1 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:19.110726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74249c84-ce9f-4a49-8a6a-f66d8f56663f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:19.111393 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74249c84-ce9f-4a49-8a6a-f66d8f56663f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:19.111674 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74249c84-ce9f-4a49-8a6a-f66d8f56663f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:19.111949 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74249c84-ce9f-4a49-8a6a-f66d8f56663f admin admin] PATH_INFO: `/v3/users/cf5bbd02dc6d4c18a78cf245cc30071c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:19.113786 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74249c84-ce9f-4a49-8a6a-f66d8f56663f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf5bbd02dc6d4c18a78cf245cc30071c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:19.115802 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74249c84-ce9f-4a49-8a6a-f66d8f56663f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:19.309555 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-74249c84-ce9f-4a49-8a6a-f66d8f56663f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf5bbd02dc6d4c18a78cf245cc30071c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:19.318523 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-74249c84-ce9f-4a49-8a6a-f66d8f56663f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf5bbd02dc6d4c18a78cf245cc30071c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:19.405604 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-74249c84-ce9f-4a49-8a6a-f66d8f56663f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf5bbd02dc6d4c18a78cf245cc30071c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:19.409986 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-74249c84-ce9f-4a49-8a6a-f66d8f56663f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf5bbd02dc6d4c18a78cf245cc30071c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:19.424394 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-74249c84-ce9f-4a49-8a6a-f66d8f56663f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf5bbd02dc6d4c18a78cf245cc30071c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:19.431316 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 979/1956] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:33:19 2022] DELETE /identity/v3/users/cf5bbd02dc6d4c18a78cf245cc30071c => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:19.701592 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:19.810241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a04d78-24d9-4569-bd71-8984cf1ff356 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:19.810806 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a04d78-24d9-4569-bd71-8984cf1ff356 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:19.810806 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a04d78-24d9-4569-bd71-8984cf1ff356 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:19.812345 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 978/1957] 10.222.0.57 () {54 vars in 921 bytes} [Fri Sep 2 09:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 09:33:19.824532 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79158be8-dc83-40fd-bb3b-d6905e3cf051 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:19.824859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79158be8-dc83-40fd-bb3b-d6905e3cf051 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:19.825085 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79158be8-dc83-40fd-bb3b-d6905e3cf051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:19.846002 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-79158be8-dc83-40fd-bb3b-d6905e3cf051 None None] MFA Rules not processed for user `1da355c8be2d496bb08feb0fe74e04b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:19.847684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-79158be8-dc83-40fd-bb3b-d6905e3cf051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:19.880669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a04d78-24d9-4569-bd71-8984cf1ff356 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:19.882676 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-81a04d78-24d9-4569-bd71-8984cf1ff356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:19.936625 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 980/1958] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Sep 2 09:33:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:19.958397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79158be8-dc83-40fd-bb3b-d6905e3cf051 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:19.960506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-79158be8-dc83-40fd-bb3b-d6905e3cf051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:19.978527 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79158be8-dc83-40fd-bb3b-d6905e3cf051 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:19.985893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0eb6db-780d-4942-913c-64966f523603 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:19.986829 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0eb6db-780d-4942-913c-64966f523603 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:19.987134 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0eb6db-780d-4942-913c-64966f523603 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:19.994485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0eb6db-780d-4942-913c-64966f523603 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:19.997707 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0eb6db-780d-4942-913c-64966f523603 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:19.999802 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0eb6db-780d-4942-913c-64966f523603 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:20.008763 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a04d78-24d9-4569-bd71-8984cf1ff356 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:20.012031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 981/1959] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:33:20.017309 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d857a3b8-5945-4ec0-8b3c-16f0064332b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:20.017962 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857a3b8-5945-4ec0-8b3c-16f0064332b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:20.018243 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857a3b8-5945-4ec0-8b3c-16f0064332b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:20.018549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857a3b8-5945-4ec0-8b3c-16f0064332b6 admin admin] PATH_INFO: `/v3/projects/a1d012bab8e24e848d7a62a92828348d` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:20.020331 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d857a3b8-5945-4ec0-8b3c-16f0064332b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1d012bab8e24e848d7a62a92828348d)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:20.022297 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d857a3b8-5945-4ec0-8b3c-16f0064332b6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:20.036308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:20.169569 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d857a3b8-5945-4ec0-8b3c-16f0064332b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1d012bab8e24e848d7a62a92828348d', 'request_id': 'req-d857a3b8-5945-4ec0-8b3c-16f0064332b6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:20.176658 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d857a3b8-5945-4ec0-8b3c-16f0064332b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1d012bab8e24e848d7a62a92828348d', 'request_id': 'req-d857a3b8-5945-4ec0-8b3c-16f0064332b6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:20.186722 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d857a3b8-5945-4ec0-8b3c-16f0064332b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1d012bab8e24e848d7a62a92828348d', 'request_id': 'req-d857a3b8-5945-4ec0-8b3c-16f0064332b6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:20.272015 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d857a3b8-5945-4ec0-8b3c-16f0064332b6 admin admin] The token cache is being invalidate because project a1d012bab8e24e848d7a62a92828348d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:33:20.272314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d857a3b8-5945-4ec0-8b3c-16f0064332b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:20.277939 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d857a3b8-5945-4ec0-8b3c-16f0064332b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:20.277939 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 979/1960] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:33:19 2022] DELETE /identity/v3/projects/a1d012bab8e24e848d7a62a92828348d => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:20.279651 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032be61-3242-4d00-9f4c-be7103c5a700 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:20.280843 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032be61-3242-4d00-9f4c-be7103c5a700 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:20.281427 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2032be61-3242-4d00-9f4c-be7103c5a700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:20.304265 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2032be61-3242-4d00-9f4c-be7103c5a700 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:20.305480 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2032be61-3242-4d00-9f4c-be7103c5a700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:20.384482 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd57d58-5224-43ce-b6e4-3e8da2612982 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:20.384653 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd57d58-5224-43ce-b6e4-3e8da2612982 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:20.385094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd57d58-5224-43ce-b6e4-3e8da2612982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:20.388618 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 982/1961] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:20.409566 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0bd57d58-5224-43ce-b6e4-3e8da2612982 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:20.411377 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0bd57d58-5224-43ce-b6e4-3e8da2612982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:20.420326 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 980/1962] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:20.424798 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2032be61-3242-4d00-9f4c-be7103c5a700 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:20.434923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2032be61-3242-4d00-9f4c-be7103c5a700 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:20.439736 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4671f9c-71bb-4c01-b345-ebbd502e687f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:20.440335 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4671f9c-71bb-4c01-b345-ebbd502e687f admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:20.441268 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4671f9c-71bb-4c01-b345-ebbd502e687f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:20.441795 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4671f9c-71bb-4c01-b345-ebbd502e687f admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:20.442448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4671f9c-71bb-4c01-b345-ebbd502e687f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:33:20.442783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4671f9c-71bb-4c01-b345-ebbd502e687f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:20.444364 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4671f9c-71bb-4c01-b345-ebbd502e687f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:20.458750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 983/1963] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:33:20.460735 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd57d58-5224-43ce-b6e4-3e8da2612982 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:20.470050 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd57d58-5224-43ce-b6e4-3e8da2612982 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:20.475350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b623dd-308c-459a-9f00-988de72d2345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:20.475966 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b623dd-308c-459a-9f00-988de72d2345 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:20.476246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b623dd-308c-459a-9f00-988de72d2345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:20.476521 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b623dd-308c-459a-9f00-988de72d2345 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:20.477158 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b623dd-308c-459a-9f00-988de72d2345 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:20.479115 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b623dd-308c-459a-9f00-988de72d2345 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:20.584082 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 981/1964] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:33:20 2022] POST /identity/v3/projects => generated 367 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:20.587808 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4671f9c-71bb-4c01-b345-ebbd502e687f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:20.598095 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4671f9c-71bb-4c01-b345-ebbd502e687f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:20.653291 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec402c5-64ef-4361-8ce3-a34ee22fae56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:20.653891 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec402c5-64ef-4361-8ce3-a34ee22fae56 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:20.654173 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec402c5-64ef-4361-8ce3-a34ee22fae56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:20.654447 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec402c5-64ef-4361-8ce3-a34ee22fae56 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:20.655142 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec402c5-64ef-4361-8ce3-a34ee22fae56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:20.657069 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec402c5-64ef-4361-8ce3-a34ee22fae56 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:20.763247 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 984/1965] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:33:20 2022] POST /identity/v3/users => generated 284 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:20.767256 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b623dd-308c-459a-9f00-988de72d2345 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:20.777975 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b623dd-308c-459a-9f00-988de72d2345 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:20.782750 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3639a-a8da-4e23-ab4e-b275a0e81c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:20.783391 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3639a-a8da-4e23-ab4e-b275a0e81c27 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:20.787357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3639a-a8da-4e23-ab4e-b275a0e81c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:20.787357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c3639a-a8da-4e23-ab4e-b275a0e81c27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:20.787357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c3639a-a8da-4e23-ab4e-b275a0e81c27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:20.787357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c3639a-a8da-4e23-ab4e-b275a0e81c27 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:20.796415 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 982/1966] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:20.800417 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec402c5-64ef-4361-8ce3-a34ee22fae56 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:20.811180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec402c5-64ef-4361-8ce3-a34ee22fae56 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:20.816038 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58120f-0ebc-44e1-af09-b0e256e30e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:20.816581 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58120f-0ebc-44e1-af09-b0e256e30e6a admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:20.817370 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58120f-0ebc-44e1-af09-b0e256e30e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:20.817641 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58120f-0ebc-44e1-af09-b0e256e30e6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:20.818279 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58120f-0ebc-44e1-af09-b0e256e30e6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:20.819907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58120f-0ebc-44e1-af09-b0e256e30e6a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:20.830646 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 985/1967] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:20.835015 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3639a-a8da-4e23-ab4e-b275a0e81c27 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:20.846621 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c3639a-a8da-4e23-ab4e-b275a0e81c27 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:20.852457 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6e2c9-73fa-4b88-a4b7-28998be51b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:20.853127 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6e2c9-73fa-4b88-a4b7-28998be51b98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:20.853428 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6e2c9-73fa-4b88-a4b7-28998be51b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:20.853746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6e2c9-73fa-4b88-a4b7-28998be51b98 admin admin] PATH_INFO: `/v3/projects/5cebcfd2bdde4adb91c68689dfc3e106/users/53b2c5a17eef43c0827b4801ea691573/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:20.869506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6e2c9-73fa-4b88-a4b7-28998be51b98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cebcfd2bdde4adb91c68689dfc3e106, user_id=53b2c5a17eef43c0827b4801ea691573, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:20.872201 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6e2c9-73fa-4b88-a4b7-28998be51b98 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:20.982233 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 983/1968] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:20 2022] PUT /identity/v3/projects/5cebcfd2bdde4adb91c68689dfc3e106/users/53b2c5a17eef43c0827b4801ea691573/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:20.993074 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bdc55-2c2b-401e-bada-238ea7d73671 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:20.993476 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bdc55-2c2b-401e-bada-238ea7d73671 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:20.993853 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bdc55-2c2b-401e-bada-238ea7d73671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:21.024199 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c02bdc55-2c2b-401e-bada-238ea7d73671 None None] MFA Rules not processed for user `53b2c5a17eef43c0827b4801ea691573`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:21.025070 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c02bdc55-2c2b-401e-bada-238ea7d73671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:21.051786 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:21.091404 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 986/1969] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:21.095913 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6e2c9-73fa-4b88-a4b7-28998be51b98 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:21.157396 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6e2c9-73fa-4b88-a4b7-28998be51b98 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:21.162797 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b6c047-8ca7-4d57-af73-b03b3b68ba3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:21.163517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b6c047-8ca7-4d57-af73-b03b3b68ba3c admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:21.163673 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b6c047-8ca7-4d57-af73-b03b3b68ba3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:21.163942 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b6c047-8ca7-4d57-af73-b03b3b68ba3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:21.164586 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b6c047-8ca7-4d57-af73-b03b3b68ba3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:21.166497 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b6c047-8ca7-4d57-af73-b03b3b68ba3c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:21.341325 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 984/1970] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:33:21 2022] POST /identity/v3/projects => generated 365 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:33:21.345356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02bdc55-2c2b-401e-bada-238ea7d73671 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:21.356780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02bdc55-2c2b-401e-bada-238ea7d73671 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:21.413741 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b376fd1-bfe5-4a0d-bd25-b1c4e185e9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:21.414392 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b376fd1-bfe5-4a0d-bd25-b1c4e185e9be admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:21.414676 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b376fd1-bfe5-4a0d-bd25-b1c4e185e9be admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:21.414982 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b376fd1-bfe5-4a0d-bd25-b1c4e185e9be admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:21.415821 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b376fd1-bfe5-4a0d-bd25-b1c4e185e9be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:21.417847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b376fd1-bfe5-4a0d-bd25-b1c4e185e9be admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:21.496490 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b6c047-8ca7-4d57-af73-b03b3b68ba3c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:21.498326 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-62b6c047-8ca7-4d57-af73-b03b3b68ba3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:21.534637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 987/1971] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:33:21 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:21.537092 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b376fd1-bfe5-4a0d-bd25-b1c4e185e9be admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:21.549910 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b376fd1-bfe5-4a0d-bd25-b1c4e185e9be admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:21.556393 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da98a3f-120e-42d2-8922-97cc84481d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:21.557185 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da98a3f-120e-42d2-8922-97cc84481d6e admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:21.557565 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da98a3f-120e-42d2-8922-97cc84481d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:21.557939 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da98a3f-120e-42d2-8922-97cc84481d6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:21.559306 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da98a3f-120e-42d2-8922-97cc84481d6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:21.562024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da98a3f-120e-42d2-8922-97cc84481d6e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:21.579413 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 988/1972] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:21.584517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da98a3f-120e-42d2-8922-97cc84481d6e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:21.595226 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da98a3f-120e-42d2-8922-97cc84481d6e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:21.601438 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f48bbc-a550-400a-bebd-72afef771c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:21.602181 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f48bbc-a550-400a-bebd-72afef771c01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:21.602528 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f48bbc-a550-400a-bebd-72afef771c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:21.602903 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f48bbc-a550-400a-bebd-72afef771c01 admin admin] PATH_INFO: `/v3/projects/6aaf7e861ff6414fbf24e38bccaec480/users/9c1fef1e29bf4df587d101011850e281/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:21.620946 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f48bbc-a550-400a-bebd-72afef771c01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aaf7e861ff6414fbf24e38bccaec480, user_id=9c1fef1e29bf4df587d101011850e281, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:21.627149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f48bbc-a550-400a-bebd-72afef771c01 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:21.631490 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b6c047-8ca7-4d57-af73-b03b3b68ba3c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:21.636131 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9700e7-bf31-4ef1-a914-17630acf983a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:21.636815 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9700e7-bf31-4ef1-a914-17630acf983a service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:21.637163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9700e7-bf31-4ef1-a914-17630acf983a service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:21.637637 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9700e7-bf31-4ef1-a914-17630acf983a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:21.639989 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8a9700e7-bf31-4ef1-a914-17630acf983a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:21.719232 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9700e7-bf31-4ef1-a914-17630acf983a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:21.723799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9700e7-bf31-4ef1-a914-17630acf983a service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:21.727087 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 989/1973] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:21 2022] PUT /identity/v3/projects/6aaf7e861ff6414fbf24e38bccaec480/users/9c1fef1e29bf4df587d101011850e281/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:33:21.730971 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f48bbc-a550-400a-bebd-72afef771c01 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:21.734600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 985/1974] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:33:21.801048 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f48bbc-a550-400a-bebd-72afef771c01 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:21.807219 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc600a9-4bc8-4c6d-b47a-355fa773d16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:21.812572 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc600a9-4bc8-4c6d-b47a-355fa773d16a admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:21.812572 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc600a9-4bc8-4c6d-b47a-355fa773d16a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:21.812572 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc600a9-4bc8-4c6d-b47a-355fa773d16a admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:21.812572 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc600a9-4bc8-4c6d-b47a-355fa773d16a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:21.812572 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc600a9-4bc8-4c6d-b47a-355fa773d16a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:21.822740 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 990/1975] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:33:21.825708 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9700e7-bf31-4ef1-a914-17630acf983a service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:21.836713 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9700e7-bf31-4ef1-a914-17630acf983a service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:21.841785 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4241bb-120c-47ed-9c13-6acaa14a9247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:21.842690 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4241bb-120c-47ed-9c13-6acaa14a9247 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:21.843134 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4241bb-120c-47ed-9c13-6acaa14a9247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:21.843542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4241bb-120c-47ed-9c13-6acaa14a9247 admin admin] PATH_INFO: `/v3/projects/6aaf7e861ff6414fbf24e38bccaec480/users/9c1fef1e29bf4df587d101011850e281/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:21.845615 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4241bb-120c-47ed-9c13-6acaa14a9247 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aaf7e861ff6414fbf24e38bccaec480, user_id=9c1fef1e29bf4df587d101011850e281, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:21.849728 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4241bb-120c-47ed-9c13-6acaa14a9247 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:21.860782 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 986/1976] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:21 2022] PUT /identity/v3/projects/6aaf7e861ff6414fbf24e38bccaec480/users/9c1fef1e29bf4df587d101011850e281/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:21.872273 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd2d0d-a519-4152-888d-7bc63b51eb5e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:21.872710 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd2d0d-a519-4152-888d-7bc63b51eb5e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:21.873266 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd2d0d-a519-4152-888d-7bc63b51eb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:21.900547 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:21.901819 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c6fd2d0d-a519-4152-888d-7bc63b51eb5e None None] MFA Rules not processed for user `9c1fef1e29bf4df587d101011850e281`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:21.903106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c6fd2d0d-a519-4152-888d-7bc63b51eb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:21.986846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 991/1977] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:21 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:22.014486 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4241bb-120c-47ed-9c13-6acaa14a9247 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:22.071699 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4241bb-120c-47ed-9c13-6acaa14a9247 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:22.075436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56012df-1ffe-48b7-b08c-30ddfee6c083 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:22.076106 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56012df-1ffe-48b7-b08c-30ddfee6c083 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:22.076413 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56012df-1ffe-48b7-b08c-30ddfee6c083 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:22.076707 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56012df-1ffe-48b7-b08c-30ddfee6c083 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:22.086601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56012df-1ffe-48b7-b08c-30ddfee6c083 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:22.088456 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56012df-1ffe-48b7-b08c-30ddfee6c083 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:22.098466 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 987/1978] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:22 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:33:22.214287 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:22.544808 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd2d0d-a519-4152-888d-7bc63b51eb5e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:22.546326 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c6fd2d0d-a519-4152-888d-7bc63b51eb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:22.558680 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd2d0d-a519-4152-888d-7bc63b51eb5e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:22.562306 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606e38a-d702-4da4-841e-15c26fc8aa0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:22.562940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606e38a-d702-4da4-841e-15c26fc8aa0d service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:22.567070 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606e38a-d702-4da4-841e-15c26fc8aa0d service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:22.567070 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606e38a-d702-4da4-841e-15c26fc8aa0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:22.618992 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606e38a-d702-4da4-841e-15c26fc8aa0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:22.620481 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606e38a-d702-4da4-841e-15c26fc8aa0d service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:22.630857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 992/1979] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:33:22.828604 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:23.022601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:23.602471 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56012df-1ffe-48b7-b08c-30ddfee6c083 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:23.604109 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f56012df-1ffe-48b7-b08c-30ddfee6c083 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:23.715505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56012df-1ffe-48b7-b08c-30ddfee6c083 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:23.720953 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff900b4e-2097-46f9-bf09-5f72bb67f9f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:23.721625 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff900b4e-2097-46f9-bf09-5f72bb67f9f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:23.721906 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff900b4e-2097-46f9-bf09-5f72bb67f9f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:23.722349 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff900b4e-2097-46f9-bf09-5f72bb67f9f5 admin admin] PATH_INFO: `/v3/users/822d68bc5c044db3a0842bde2fdf1a73` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:23.724680 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff900b4e-2097-46f9-bf09-5f72bb67f9f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=822d68bc5c044db3a0842bde2fdf1a73)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:23.727161 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff900b4e-2097-46f9-bf09-5f72bb67f9f5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:23.855366 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:23.964665 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ff900b4e-2097-46f9-bf09-5f72bb67f9f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '822d68bc5c044db3a0842bde2fdf1a73'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:24.010914 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ff900b4e-2097-46f9-bf09-5f72bb67f9f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '822d68bc5c044db3a0842bde2fdf1a73'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:24.011724 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ff900b4e-2097-46f9-bf09-5f72bb67f9f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '822d68bc5c044db3a0842bde2fdf1a73'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:24.012521 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ff900b4e-2097-46f9-bf09-5f72bb67f9f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '822d68bc5c044db3a0842bde2fdf1a73'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:24.025828 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ff900b4e-2097-46f9-bf09-5f72bb67f9f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '822d68bc5c044db3a0842bde2fdf1a73'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:24.038298 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 988/1980] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:33:23 2022] DELETE /identity/v3/users/822d68bc5c044db3a0842bde2fdf1a73 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:24.043152 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab16ef8-928d-4a79-9de6-feade611649f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:24.043587 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab16ef8-928d-4a79-9de6-feade611649f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:24.043775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab16ef8-928d-4a79-9de6-feade611649f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:24.066949 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:24.072682 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6ab16ef8-928d-4a79-9de6-feade611649f None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:24.073896 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6ab16ef8-928d-4a79-9de6-feade611649f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:24.142390 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 993/1981] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:33:24.153207 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff900b4e-2097-46f9-bf09-5f72bb67f9f5 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:24.155011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ff900b4e-2097-46f9-bf09-5f72bb67f9f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:24.214238 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff900b4e-2097-46f9-bf09-5f72bb67f9f5 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:24.217824 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c135d6d-82c4-4bbb-bf71-5b35399914f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:24.218478 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c135d6d-82c4-4bbb-bf71-5b35399914f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:24.218773 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c135d6d-82c4-4bbb-bf71-5b35399914f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:24.219048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c135d6d-82c4-4bbb-bf71-5b35399914f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:24.228297 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c135d6d-82c4-4bbb-bf71-5b35399914f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:24.229747 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c135d6d-82c4-4bbb-bf71-5b35399914f4 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:24.239588 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 989/1982] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:33:24.562804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:24.666769 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab16ef8-928d-4a79-9de6-feade611649f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:24.668224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6ab16ef8-928d-4a79-9de6-feade611649f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:24.680448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab16ef8-928d-4a79-9de6-feade611649f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:24.687595 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd970c72-5204-485f-b50c-89b1f5dfdb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:24.688342 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd970c72-5204-485f-b50c-89b1f5dfdb97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:24.688511 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd970c72-5204-485f-b50c-89b1f5dfdb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:24.689121 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd970c72-5204-485f-b50c-89b1f5dfdb97 admin admin] PATH_INFO: `/v3/projects/f8dea0a90aaf402da76b76f721da3878` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:24.701152 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd970c72-5204-485f-b50c-89b1f5dfdb97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8dea0a90aaf402da76b76f721da3878)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:24.704303 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd970c72-5204-485f-b50c-89b1f5dfdb97 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:24.817739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dd970c72-5204-485f-b50c-89b1f5dfdb97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8dea0a90aaf402da76b76f721da3878', 'request_id': 'req-dd970c72-5204-485f-b50c-89b1f5dfdb97'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:24.962999 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dd970c72-5204-485f-b50c-89b1f5dfdb97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8dea0a90aaf402da76b76f721da3878', 'request_id': 'req-dd970c72-5204-485f-b50c-89b1f5dfdb97'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:24.969258 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dd970c72-5204-485f-b50c-89b1f5dfdb97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8dea0a90aaf402da76b76f721da3878', 'request_id': 'req-dd970c72-5204-485f-b50c-89b1f5dfdb97'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:24.981718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dd970c72-5204-485f-b50c-89b1f5dfdb97 admin admin] The token cache is being invalidate because project f8dea0a90aaf402da76b76f721da3878 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:33:24.982027 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dd970c72-5204-485f-b50c-89b1f5dfdb97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:24.982643 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dd970c72-5204-485f-b50c-89b1f5dfdb97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:24.984175 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 994/1983] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:33:24 2022] DELETE /identity/v3/projects/f8dea0a90aaf402da76b76f721da3878 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:25.106541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c135d6d-82c4-4bbb-bf71-5b35399914f4 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:25.113236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c135d6d-82c4-4bbb-bf71-5b35399914f4 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:25.172076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90206a22-ffd7-4469-8e60-e2dd76c51285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:25.172760 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90206a22-ffd7-4469-8e60-e2dd76c51285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:25.173047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90206a22-ffd7-4469-8e60-e2dd76c51285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:25.173321 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90206a22-ffd7-4469-8e60-e2dd76c51285 admin admin] PATH_INFO: `/v3/users/9fff8e743d484bc686717cfbe716329f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:25.174462 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90206a22-ffd7-4469-8e60-e2dd76c51285 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fff8e743d484bc686717cfbe716329f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:25.176453 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90206a22-ffd7-4469-8e60-e2dd76c51285 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:25.254935 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:25.790872 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-90206a22-ffd7-4469-8e60-e2dd76c51285 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fff8e743d484bc686717cfbe716329f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:25.885424 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-90206a22-ffd7-4469-8e60-e2dd76c51285 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fff8e743d484bc686717cfbe716329f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:25.897488 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-90206a22-ffd7-4469-8e60-e2dd76c51285 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fff8e743d484bc686717cfbe716329f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:25.898212 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-90206a22-ffd7-4469-8e60-e2dd76c51285 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fff8e743d484bc686717cfbe716329f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:25.931540 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-90206a22-ffd7-4469-8e60-e2dd76c51285 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fff8e743d484bc686717cfbe716329f'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:25.946272 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 990/1984] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:33:25 2022] DELETE /identity/v3/users/9fff8e743d484bc686717cfbe716329f => generated 0 bytes in 844 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:26.131417 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:26.845963 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:27.061518 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd970c72-5204-485f-b50c-89b1f5dfdb97 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:27.063064 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-dd970c72-5204-485f-b50c-89b1f5dfdb97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:27.076544 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd970c72-5204-485f-b50c-89b1f5dfdb97 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:27.135211 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f852d5-3a83-4d1d-b655-e0ad389873eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:27.135839 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f852d5-3a83-4d1d-b655-e0ad389873eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:27.136120 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f852d5-3a83-4d1d-b655-e0ad389873eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:27.136390 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f852d5-3a83-4d1d-b655-e0ad389873eb admin admin] PATH_INFO: `/v3/projects/ba3f6ec8d61849808f6990dcb92aed27` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:27.137486 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f852d5-3a83-4d1d-b655-e0ad389873eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba3f6ec8d61849808f6990dcb92aed27)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:27.139426 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f852d5-3a83-4d1d-b655-e0ad389873eb admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:27.246642 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-86f852d5-3a83-4d1d-b655-e0ad389873eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba3f6ec8d61849808f6990dcb92aed27', 'request_id': 'req-86f852d5-3a83-4d1d-b655-e0ad389873eb'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:27.339830 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-86f852d5-3a83-4d1d-b655-e0ad389873eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba3f6ec8d61849808f6990dcb92aed27', 'request_id': 'req-86f852d5-3a83-4d1d-b655-e0ad389873eb'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:27.345617 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-86f852d5-3a83-4d1d-b655-e0ad389873eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba3f6ec8d61849808f6990dcb92aed27', 'request_id': 'req-86f852d5-3a83-4d1d-b655-e0ad389873eb'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:27.356523 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-86f852d5-3a83-4d1d-b655-e0ad389873eb admin admin] The token cache is being invalidate because project ba3f6ec8d61849808f6990dcb92aed27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:33:27.356813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-86f852d5-3a83-4d1d-b655-e0ad389873eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:27.357437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-86f852d5-3a83-4d1d-b655-e0ad389873eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:27.359052 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 995/1985] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:33:27 2022] DELETE /identity/v3/projects/ba3f6ec8d61849808f6990dcb92aed27 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:27.363926 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f8605-2301-4af6-9a4e-2d17e43eec5e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:27.366665 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f8605-2301-4af6-9a4e-2d17e43eec5e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:27.366665 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f8605-2301-4af6-9a4e-2d17e43eec5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:27.385727 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1b3f8605-2301-4af6-9a4e-2d17e43eec5e None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:27.387198 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1b3f8605-2301-4af6-9a4e-2d17e43eec5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:27.447248 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 991/1986] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:27.454801 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac63ca6-d097-4a62-930f-a9597733e01e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:27.455089 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac63ca6-d097-4a62-930f-a9597733e01e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:27.455358 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac63ca6-d097-4a62-930f-a9597733e01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:27.474864 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-fac63ca6-d097-4a62-930f-a9597733e01e None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:27.478947 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fac63ca6-d097-4a62-930f-a9597733e01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:27.483890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 996/1987] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:27.488020 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f8605-2301-4af6-9a4e-2d17e43eec5e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:27.498047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f8605-2301-4af6-9a4e-2d17e43eec5e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:27.502766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a95b19-2a4e-4a0a-a791-a32b2efffe53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:27.503375 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a95b19-2a4e-4a0a-a791-a32b2efffe53 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:27.503656 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a95b19-2a4e-4a0a-a791-a32b2efffe53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:27.504191 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a95b19-2a4e-4a0a-a791-a32b2efffe53 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:27.504843 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a95b19-2a4e-4a0a-a791-a32b2efffe53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:33:27.505142 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a95b19-2a4e-4a0a-a791-a32b2efffe53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:27.506743 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a95b19-2a4e-4a0a-a791-a32b2efffe53 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:27.518844 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 992/1988] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:33:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:33:27.523209 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac63ca6-d097-4a62-930f-a9597733e01e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:27.532334 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac63ca6-d097-4a62-930f-a9597733e01e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:27.536997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681e9a2-d47f-4ed4-bd5c-b95f663f82dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:27.537591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681e9a2-d47f-4ed4-bd5c-b95f663f82dd admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:27.537874 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681e9a2-d47f-4ed4-bd5c-b95f663f82dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:27.538146 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681e9a2-d47f-4ed4-bd5c-b95f663f82dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:27.538832 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6681e9a2-d47f-4ed4-bd5c-b95f663f82dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:27.540717 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6681e9a2-d47f-4ed4-bd5c-b95f663f82dd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:27.553826 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:27.641109 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 997/1989] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:33:27 2022] POST /identity/v3/projects => generated 363 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:27.646872 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a95b19-2a4e-4a0a-a791-a32b2efffe53 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:27.655737 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a95b19-2a4e-4a0a-a791-a32b2efffe53 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:27.719194 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c322ad-f9c7-4e99-a232-ac54b83141c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:27.719931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c322ad-f9c7-4e99-a232-ac54b83141c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:27.720287 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c322ad-f9c7-4e99-a232-ac54b83141c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:27.720632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c322ad-f9c7-4e99-a232-ac54b83141c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:27.721339 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c322ad-f9c7-4e99-a232-ac54b83141c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:27.723538 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c322ad-f9c7-4e99-a232-ac54b83141c2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:27.823777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 993/1990] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:33:27 2022] POST /identity/v3/users => generated 282 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:27.824848 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681e9a2-d47f-4ed4-bd5c-b95f663f82dd admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:27.836526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681e9a2-d47f-4ed4-bd5c-b95f663f82dd admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:27.844123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cbec04-5a77-477f-b678-08b353533cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:27.844795 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbec04-5a77-477f-b678-08b353533cc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:27.845085 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbec04-5a77-477f-b678-08b353533cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:27.845363 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cbec04-5a77-477f-b678-08b353533cc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:27.846039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cbec04-5a77-477f-b678-08b353533cc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:27.847711 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cbec04-5a77-477f-b678-08b353533cc4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:27.858769 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 998/1991] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:27.862819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c322ad-f9c7-4e99-a232-ac54b83141c2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:27.873396 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c322ad-f9c7-4e99-a232-ac54b83141c2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:27.878029 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a8d89-cb22-4929-8b87-6a6a520fdc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:27.878641 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a8d89-cb22-4929-8b87-6a6a520fdc75 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:27.879201 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a8d89-cb22-4929-8b87-6a6a520fdc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:27.879477 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a8d89-cb22-4929-8b87-6a6a520fdc75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:27.880122 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743a8d89-cb22-4929-8b87-6a6a520fdc75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:27.881752 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743a8d89-cb22-4929-8b87-6a6a520fdc75 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:27.892626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 994/1992] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:27.896635 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cbec04-5a77-477f-b678-08b353533cc4 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:27.906444 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cbec04-5a77-477f-b678-08b353533cc4 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:27.911769 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eda909-2db7-4dfd-92a6-cd7216d1aa68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:27.912478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eda909-2db7-4dfd-92a6-cd7216d1aa68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:27.912770 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eda909-2db7-4dfd-92a6-cd7216d1aa68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:27.913047 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eda909-2db7-4dfd-92a6-cd7216d1aa68 admin admin] PATH_INFO: `/v3/projects/365579494a76431385d168c0093ffa2e/users/55fd2d2f963b47239091aa5bb6e3fbaa/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:27.929428 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eda909-2db7-4dfd-92a6-cd7216d1aa68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=365579494a76431385d168c0093ffa2e, user_id=55fd2d2f963b47239091aa5bb6e3fbaa, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:27.932895 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3eda909-2db7-4dfd-92a6-cd7216d1aa68 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:28.010605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 999/1993] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:27 2022] PUT /identity/v3/projects/365579494a76431385d168c0093ffa2e/users/55fd2d2f963b47239091aa5bb6e3fbaa/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:28.016397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:28.020898 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586eefd-cdf8-419a-8e49-8eb247168c8c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:28.021187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586eefd-cdf8-419a-8e49-8eb247168c8c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:28.021468 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586eefd-cdf8-419a-8e49-8eb247168c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:28.048652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2586eefd-cdf8-419a-8e49-8eb247168c8c None None] MFA Rules not processed for user `55fd2d2f963b47239091aa5bb6e3fbaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:28.049879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2586eefd-cdf8-419a-8e49-8eb247168c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:28.124433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 995/1994] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:28.128615 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eda909-2db7-4dfd-92a6-cd7216d1aa68 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:28.182368 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:28.191955 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eda909-2db7-4dfd-92a6-cd7216d1aa68 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:28.196939 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f24e74-fed7-4a5f-b5fb-dd3713562572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:28.197544 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f24e74-fed7-4a5f-b5fb-dd3713562572 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:28.197822 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f24e74-fed7-4a5f-b5fb-dd3713562572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:28.198092 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f24e74-fed7-4a5f-b5fb-dd3713562572 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:28.198794 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f24e74-fed7-4a5f-b5fb-dd3713562572 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:28.200718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f24e74-fed7-4a5f-b5fb-dd3713562572 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:28.233630 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:28.303255 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1000/1995] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:33:28 2022] POST /identity/v3/projects => generated 365 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:28.307966 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586eefd-cdf8-419a-8e49-8eb247168c8c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:28.318621 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586eefd-cdf8-419a-8e49-8eb247168c8c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:28.373162 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b5b60-71ea-402d-9817-49ac9aa4333c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:28.373790 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b5b60-71ea-402d-9817-49ac9aa4333c admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:28.374045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b5b60-71ea-402d-9817-49ac9aa4333c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:28.374321 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3b5b60-71ea-402d-9817-49ac9aa4333c admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:28.375003 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3b5b60-71ea-402d-9817-49ac9aa4333c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:28.376908 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3b5b60-71ea-402d-9817-49ac9aa4333c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:28.482764 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 996/1996] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:33:28 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:28.486622 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f24e74-fed7-4a5f-b5fb-dd3713562572 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:28.496690 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f24e74-fed7-4a5f-b5fb-dd3713562572 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:28.501360 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283f1c4-7514-4a7c-9d3c-3822a4f73858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:28.501951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283f1c4-7514-4a7c-9d3c-3822a4f73858 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:28.502224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283f1c4-7514-4a7c-9d3c-3822a4f73858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:28.502490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283f1c4-7514-4a7c-9d3c-3822a4f73858 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:28.503165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283f1c4-7514-4a7c-9d3c-3822a4f73858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:28.505324 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283f1c4-7514-4a7c-9d3c-3822a4f73858 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:28.516871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1001/1997] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:28.521009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b5b60-71ea-402d-9817-49ac9aa4333c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:28.531390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3b5b60-71ea-402d-9817-49ac9aa4333c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:28.534329 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:28.537496 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aaa550-469c-4716-9a7b-2e3c474750fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:28.543329 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aaa550-469c-4716-9a7b-2e3c474750fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:28.543329 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aaa550-469c-4716-9a7b-2e3c474750fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:28.543329 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aaa550-469c-4716-9a7b-2e3c474750fa admin admin] PATH_INFO: `/v3/projects/be6c8a0f0be44c87aa6c09bd434cdfba/users/5ec6937e437d45b2bfb6f111aeff2592/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:28.553698 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aaa550-469c-4716-9a7b-2e3c474750fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=be6c8a0f0be44c87aa6c09bd434cdfba, user_id=5ec6937e437d45b2bfb6f111aeff2592, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:28.556346 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aaa550-469c-4716-9a7b-2e3c474750fa admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:28.637218 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 997/1998] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:28 2022] PUT /identity/v3/projects/be6c8a0f0be44c87aa6c09bd434cdfba/users/5ec6937e437d45b2bfb6f111aeff2592/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:28.641128 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283f1c4-7514-4a7c-9d3c-3822a4f73858 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:28.651453 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283f1c4-7514-4a7c-9d3c-3822a4f73858 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:28.701006 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:28.705989 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c34240-6488-49e2-97d3-47f6a6060b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:28.709832 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c34240-6488-49e2-97d3-47f6a6060b7c admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:28.709832 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c34240-6488-49e2-97d3-47f6a6060b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:28.709832 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c34240-6488-49e2-97d3-47f6a6060b7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:28.709832 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c34240-6488-49e2-97d3-47f6a6060b7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:28.709832 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c34240-6488-49e2-97d3-47f6a6060b7c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:28.719547 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1002/1999] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:28.723787 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aaa550-469c-4716-9a7b-2e3c474750fa admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:28.733824 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aaa550-469c-4716-9a7b-2e3c474750fa admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:28.736140 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:28.739902 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7d5993-bef7-457c-8594-f48f22b52826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:28.744334 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d5993-bef7-457c-8594-f48f22b52826 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:28.744334 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d5993-bef7-457c-8594-f48f22b52826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:28.744334 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d5993-bef7-457c-8594-f48f22b52826 admin admin] PATH_INFO: `/v3/projects/be6c8a0f0be44c87aa6c09bd434cdfba/users/5ec6937e437d45b2bfb6f111aeff2592/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:28.744334 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7d5993-bef7-457c-8594-f48f22b52826 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be6c8a0f0be44c87aa6c09bd434cdfba, user_id=5ec6937e437d45b2bfb6f111aeff2592, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:28.745680 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7d5993-bef7-457c-8594-f48f22b52826 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:28.755837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 998/2000] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:28 2022] PUT /identity/v3/projects/be6c8a0f0be44c87aa6c09bd434cdfba/users/5ec6937e437d45b2bfb6f111aeff2592/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:28.767444 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cc91a-1416-4815-b7eb-8410c725b4ea None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:28.767987 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cc91a-1416-4815-b7eb-8410c725b4ea None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:28.768525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cc91a-1416-4815-b7eb-8410c725b4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:28.796740 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1b6cc91a-1416-4815-b7eb-8410c725b4ea None None] MFA Rules not processed for user `5ec6937e437d45b2bfb6f111aeff2592`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:28.797942 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1b6cc91a-1416-4815-b7eb-8410c725b4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:28.870777 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1003/2001] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:28 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:33:28.883028 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7d5993-bef7-457c-8594-f48f22b52826 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:28.884801 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7a7d5993-bef7-457c-8594-f48f22b52826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:28.907441 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:28.937300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:28.950239 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7d5993-bef7-457c-8594-f48f22b52826 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:28.954127 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95744da1-ddd1-456c-ba95-37e0bff4693a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:28.954901 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95744da1-ddd1-456c-ba95-37e0bff4693a service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:28.955420 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95744da1-ddd1-456c-ba95-37e0bff4693a service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:28.955767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95744da1-ddd1-456c-ba95-37e0bff4693a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:28.970624 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95744da1-ddd1-456c-ba95-37e0bff4693a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:28.970624 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95744da1-ddd1-456c-ba95-37e0bff4693a service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:28.980617 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 999/2002] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:28 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:33:29.050859 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:29.127298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6cc91a-1416-4815-b7eb-8410c725b4ea None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:29.138164 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6cc91a-1416-4815-b7eb-8410c725b4ea None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:29.141897 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab5b90c-dd75-4dd6-88a3-3568fec670c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:29.142507 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5b90c-dd75-4dd6-88a3-3568fec670c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:29.142820 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5b90c-dd75-4dd6-88a3-3568fec670c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:29.143096 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab5b90c-dd75-4dd6-88a3-3568fec670c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:29.167848 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:29.201281 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5b90c-dd75-4dd6-88a3-3568fec670c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:29.202826 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab5b90c-dd75-4dd6-88a3-3568fec670c7 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:29.211728 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1004/2003] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:33:29.285256 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:29.340074 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:29.405238 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:29.631724 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:29.799288 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:29.833804 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:30.142397 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:30.225358 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:30.488493 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:30.547702 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:30.931039 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:31.005782 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:31.522862 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:31.564007 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:32.338326 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:33.022239 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:33.909725 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95744da1-ddd1-456c-ba95-37e0bff4693a service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:34.005160 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:34.035904 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95744da1-ddd1-456c-ba95-37e0bff4693a service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:34.048587 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83439a4a-0fd7-4c48-aa6f-6d2beae05327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:34.049262 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83439a4a-0fd7-4c48-aa6f-6d2beae05327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:34.049823 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83439a4a-0fd7-4c48-aa6f-6d2beae05327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:34.050115 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83439a4a-0fd7-4c48-aa6f-6d2beae05327 admin admin] PATH_INFO: `/v3/users/55fd2d2f963b47239091aa5bb6e3fbaa` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:34.052027 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83439a4a-0fd7-4c48-aa6f-6d2beae05327 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55fd2d2f963b47239091aa5bb6e3fbaa)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:34.054937 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83439a4a-0fd7-4c48-aa6f-6d2beae05327 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:34.083385 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:34.365090 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-83439a4a-0fd7-4c48-aa6f-6d2beae05327 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55fd2d2f963b47239091aa5bb6e3fbaa'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:34.448728 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-83439a4a-0fd7-4c48-aa6f-6d2beae05327 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55fd2d2f963b47239091aa5bb6e3fbaa'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:34.449889 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-83439a4a-0fd7-4c48-aa6f-6d2beae05327 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55fd2d2f963b47239091aa5bb6e3fbaa'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:34.450529 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-83439a4a-0fd7-4c48-aa6f-6d2beae05327 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55fd2d2f963b47239091aa5bb6e3fbaa'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:34.465328 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-83439a4a-0fd7-4c48-aa6f-6d2beae05327 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55fd2d2f963b47239091aa5bb6e3fbaa'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:34.479066 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1000/2004] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:33:33 2022] DELETE /identity/v3/users/55fd2d2f963b47239091aa5bb6e3fbaa => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:34.483670 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82ef04-5e87-4731-ba16-bbee6aa9458d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:34.484059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82ef04-5e87-4731-ba16-bbee6aa9458d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:34.484437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82ef04-5e87-4731-ba16-bbee6aa9458d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:34.506174 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-fe82ef04-5e87-4731-ba16-bbee6aa9458d None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:34.507904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fe82ef04-5e87-4731-ba16-bbee6aa9458d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:34.572490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1005/2005] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:34.584578 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83439a4a-0fd7-4c48-aa6f-6d2beae05327 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:34.587445 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-83439a4a-0fd7-4c48-aa6f-6d2beae05327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:34.655812 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83439a4a-0fd7-4c48-aa6f-6d2beae05327 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:34.659666 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2aade9-118e-42bb-bd30-58903cc3b590 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:34.661889 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2aade9-118e-42bb-bd30-58903cc3b590 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:34.661889 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2aade9-118e-42bb-bd30-58903cc3b590 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:34.661889 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2aade9-118e-42bb-bd30-58903cc3b590 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:34.671226 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2aade9-118e-42bb-bd30-58903cc3b590 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:34.672196 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2aade9-118e-42bb-bd30-58903cc3b590 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:34.682213 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1001/2006] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:33:35.309853 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82ef04-5e87-4731-ba16-bbee6aa9458d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:35.314640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fe82ef04-5e87-4731-ba16-bbee6aa9458d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:35.324061 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82ef04-5e87-4731-ba16-bbee6aa9458d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:35.328926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da78d72-65b5-43eb-9d40-450169094b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:35.329572 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da78d72-65b5-43eb-9d40-450169094b54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:35.329854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da78d72-65b5-43eb-9d40-450169094b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:35.330452 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da78d72-65b5-43eb-9d40-450169094b54 admin admin] PATH_INFO: `/v3/projects/365579494a76431385d168c0093ffa2e` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:35.331668 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da78d72-65b5-43eb-9d40-450169094b54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=365579494a76431385d168c0093ffa2e)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:35.333617 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da78d72-65b5-43eb-9d40-450169094b54 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:35.480651 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1da78d72-65b5-43eb-9d40-450169094b54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '365579494a76431385d168c0093ffa2e', 'request_id': 'req-1da78d72-65b5-43eb-9d40-450169094b54'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:35.575889 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:35.621239 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1da78d72-65b5-43eb-9d40-450169094b54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '365579494a76431385d168c0093ffa2e', 'request_id': 'req-1da78d72-65b5-43eb-9d40-450169094b54'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:35.628731 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1da78d72-65b5-43eb-9d40-450169094b54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '365579494a76431385d168c0093ffa2e', 'request_id': 'req-1da78d72-65b5-43eb-9d40-450169094b54'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:35.641257 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1da78d72-65b5-43eb-9d40-450169094b54 admin admin] The token cache is being invalidate because project 365579494a76431385d168c0093ffa2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:33:35.641540 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1da78d72-65b5-43eb-9d40-450169094b54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:35.642262 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1da78d72-65b5-43eb-9d40-450169094b54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:35.648703 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1006/2007] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:33:35 2022] DELETE /identity/v3/projects/365579494a76431385d168c0093ffa2e => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:35.648810 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:35.649223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2aade9-118e-42bb-bd30-58903cc3b590 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:35.658855 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2aade9-118e-42bb-bd30-58903cc3b590 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:35.669289 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da78d72-65b5-43eb-9d40-450169094b54 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:35.683885 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1da78d72-65b5-43eb-9d40-450169094b54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:35.719665 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233934df-7232-49fa-96f1-d1ef3e8f4b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:35.720573 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233934df-7232-49fa-96f1-d1ef3e8f4b1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:35.720849 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233934df-7232-49fa-96f1-d1ef3e8f4b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:35.721268 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233934df-7232-49fa-96f1-d1ef3e8f4b1c admin admin] PATH_INFO: `/v3/users/5ec6937e437d45b2bfb6f111aeff2592` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:35.722434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233934df-7232-49fa-96f1-d1ef3e8f4b1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ec6937e437d45b2bfb6f111aeff2592)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:35.724463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233934df-7232-49fa-96f1-d1ef3e8f4b1c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:35.759122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da78d72-65b5-43eb-9d40-450169094b54 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:35.763963 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b3406-5a81-4b62-a07d-d006960411d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:35.764607 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b3406-5a81-4b62-a07d-d006960411d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:35.764893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b3406-5a81-4b62-a07d-d006960411d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:35.765182 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8b3406-5a81-4b62-a07d-d006960411d9 admin admin] PATH_INFO: `/v3/users/53b2c5a17eef43c0827b4801ea691573` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:35.766356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b3406-5a81-4b62-a07d-d006960411d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53b2c5a17eef43c0827b4801ea691573)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:35.768383 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8b3406-5a81-4b62-a07d-d006960411d9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:35.941617 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-233934df-7232-49fa-96f1-d1ef3e8f4b1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ec6937e437d45b2bfb6f111aeff2592'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:35.962319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6e8b3406-5a81-4b62-a07d-d006960411d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53b2c5a17eef43c0827b4801ea691573'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:35.970903 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6e8b3406-5a81-4b62-a07d-d006960411d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53b2c5a17eef43c0827b4801ea691573'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:36.021462 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-233934df-7232-49fa-96f1-d1ef3e8f4b1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ec6937e437d45b2bfb6f111aeff2592'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:36.022132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-233934df-7232-49fa-96f1-d1ef3e8f4b1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ec6937e437d45b2bfb6f111aeff2592'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:36.022840 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-233934df-7232-49fa-96f1-d1ef3e8f4b1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ec6937e437d45b2bfb6f111aeff2592'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:36.035462 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-233934df-7232-49fa-96f1-d1ef3e8f4b1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ec6937e437d45b2bfb6f111aeff2592'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:36.048210 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1002/2008] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:33:35 2022] DELETE /identity/v3/users/5ec6937e437d45b2bfb6f111aeff2592 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:36.111038 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6e8b3406-5a81-4b62-a07d-d006960411d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53b2c5a17eef43c0827b4801ea691573'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:36.112072 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6e8b3406-5a81-4b62-a07d-d006960411d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53b2c5a17eef43c0827b4801ea691573'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:36.126189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6e8b3406-5a81-4b62-a07d-d006960411d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53b2c5a17eef43c0827b4801ea691573'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:36.133311 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1007/2009] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:33:35 2022] DELETE /identity/v3/users/53b2c5a17eef43c0827b4801ea691573 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:36.137782 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208f491-c46c-4f60-bb5d-0a2332197fc7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:36.138426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208f491-c46c-4f60-bb5d-0a2332197fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:36.139753 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208f491-c46c-4f60-bb5d-0a2332197fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:36.161399 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8208f491-c46c-4f60-bb5d-0a2332197fc7 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:36.162921 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8208f491-c46c-4f60-bb5d-0a2332197fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:36.226649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1003/2010] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:36.236493 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b3406-5a81-4b62-a07d-d006960411d9 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:36.238573 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6e8b3406-5a81-4b62-a07d-d006960411d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:36.271328 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:36.332970 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8b3406-5a81-4b62-a07d-d006960411d9 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:36.337209 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98337b2d-5fcc-4841-be36-f7a25cf35aec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:36.337828 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98337b2d-5fcc-4841-be36-f7a25cf35aec service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:36.338112 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98337b2d-5fcc-4841-be36-f7a25cf35aec service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:36.338389 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98337b2d-5fcc-4841-be36-f7a25cf35aec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:36.348356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98337b2d-5fcc-4841-be36-f7a25cf35aec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:36.350089 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98337b2d-5fcc-4841-be36-f7a25cf35aec service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:36.360690 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1008/2011] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:33:36.623632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208f491-c46c-4f60-bb5d-0a2332197fc7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:36.625282 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8208f491-c46c-4f60-bb5d-0a2332197fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:36.637888 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208f491-c46c-4f60-bb5d-0a2332197fc7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:36.645477 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b88190-e30b-4407-a3b3-a143f712aad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:36.649820 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b88190-e30b-4407-a3b3-a143f712aad6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:36.649820 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b88190-e30b-4407-a3b3-a143f712aad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:36.649820 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b88190-e30b-4407-a3b3-a143f712aad6 admin admin] PATH_INFO: `/v3/projects/be6c8a0f0be44c87aa6c09bd434cdfba` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:36.649820 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b88190-e30b-4407-a3b3-a143f712aad6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be6c8a0f0be44c87aa6c09bd434cdfba)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:36.651113 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b88190-e30b-4407-a3b3-a143f712aad6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:36.795942 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-83b88190-e30b-4407-a3b3-a143f712aad6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be6c8a0f0be44c87aa6c09bd434cdfba', 'request_id': 'req-83b88190-e30b-4407-a3b3-a143f712aad6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:36.801374 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-83b88190-e30b-4407-a3b3-a143f712aad6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be6c8a0f0be44c87aa6c09bd434cdfba', 'request_id': 'req-83b88190-e30b-4407-a3b3-a143f712aad6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:36.808323 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-83b88190-e30b-4407-a3b3-a143f712aad6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be6c8a0f0be44c87aa6c09bd434cdfba', 'request_id': 'req-83b88190-e30b-4407-a3b3-a143f712aad6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:36.899027 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:36.903395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-83b88190-e30b-4407-a3b3-a143f712aad6 admin admin] The token cache is being invalidate because project be6c8a0f0be44c87aa6c09bd434cdfba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:33:36.903687 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-83b88190-e30b-4407-a3b3-a143f712aad6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:36.904023 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98337b2d-5fcc-4841-be36-f7a25cf35aec service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:36.904344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-83b88190-e30b-4407-a3b3-a143f712aad6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:36.905688 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-98337b2d-5fcc-4841-be36-f7a25cf35aec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:36.906092 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1004/2012] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:33:36 2022] DELETE /identity/v3/projects/be6c8a0f0be44c87aa6c09bd434cdfba => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:36.910825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc710b42-54b1-4194-a3dc-7eef31a64eee None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:36.911639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc710b42-54b1-4194-a3dc-7eef31a64eee None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:36.912809 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc710b42-54b1-4194-a3dc-7eef31a64eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:36.919808 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98337b2d-5fcc-4841-be36-f7a25cf35aec service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:36.950927 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-cc710b42-54b1-4194-a3dc-7eef31a64eee None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:36.952420 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cc710b42-54b1-4194-a3dc-7eef31a64eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:36.981853 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b568522-f089-4275-acfb-a6176fddf6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:36.982496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b568522-f089-4275-acfb-a6176fddf6fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:36.982928 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b568522-f089-4275-acfb-a6176fddf6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:36.983353 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b568522-f089-4275-acfb-a6176fddf6fb admin admin] PATH_INFO: `/v3/projects/5cebcfd2bdde4adb91c68689dfc3e106` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:36.984555 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b568522-f089-4275-acfb-a6176fddf6fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cebcfd2bdde4adb91c68689dfc3e106)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:36.986653 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b568522-f089-4275-acfb-a6176fddf6fb admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:37.020031 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1005/2013] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:33:37.028367 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0704f0-25e7-4b1e-9978-03d1b2dba0d5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:37.030695 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0704f0-25e7-4b1e-9978-03d1b2dba0d5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:37.031857 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0704f0-25e7-4b1e-9978-03d1b2dba0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:37.065306 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-eb0704f0-25e7-4b1e-9978-03d1b2dba0d5 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:37.067401 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-eb0704f0-25e7-4b1e-9978-03d1b2dba0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:37.078855 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1006/2014] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:37.082821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0704f0-25e7-4b1e-9978-03d1b2dba0d5 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:37.094697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0704f0-25e7-4b1e-9978-03d1b2dba0d5 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:37.101005 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d1ab8-313c-4182-a6c1-32ab44e25ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:37.101609 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d1ab8-313c-4182-a6c1-32ab44e25ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:37.101963 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d1ab8-313c-4182-a6c1-32ab44e25ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:37.102725 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d1ab8-313c-4182-a6c1-32ab44e25ce5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:37.106762 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d1ab8-313c-4182-a6c1-32ab44e25ce5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:33:37.106762 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d1ab8-313c-4182-a6c1-32ab44e25ce5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:37.106762 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d1ab8-313c-4182-a6c1-32ab44e25ce5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:37.120629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1007/2015] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:33:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:33:37.125034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d1ab8-313c-4182-a6c1-32ab44e25ce5 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:37.136339 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d1ab8-313c-4182-a6c1-32ab44e25ce5 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:37.147674 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aed5a7-69ef-4717-862c-7da8b3af8849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:37.148988 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aed5a7-69ef-4717-862c-7da8b3af8849 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:37.149464 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aed5a7-69ef-4717-862c-7da8b3af8849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:37.149748 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aed5a7-69ef-4717-862c-7da8b3af8849 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:37.152304 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aed5a7-69ef-4717-862c-7da8b3af8849 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:37.154284 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aed5a7-69ef-4717-862c-7da8b3af8849 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:37.163033 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6b568522-f089-4275-acfb-a6176fddf6fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cebcfd2bdde4adb91c68689dfc3e106', 'request_id': 'req-6b568522-f089-4275-acfb-a6176fddf6fb'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:37.262930 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6b568522-f089-4275-acfb-a6176fddf6fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cebcfd2bdde4adb91c68689dfc3e106', 'request_id': 'req-6b568522-f089-4275-acfb-a6176fddf6fb'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:37.263806 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1008/2016] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:33:37 2022] POST /identity/v3/projects => generated 387 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:37.266852 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aed5a7-69ef-4717-862c-7da8b3af8849 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:37.275009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6b568522-f089-4275-acfb-a6176fddf6fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cebcfd2bdde4adb91c68689dfc3e106', 'request_id': 'req-6b568522-f089-4275-acfb-a6176fddf6fb'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:37.292516 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6b568522-f089-4275-acfb-a6176fddf6fb admin admin] The token cache is being invalidate because project 5cebcfd2bdde4adb91c68689dfc3e106 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:33:37.296455 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6b568522-f089-4275-acfb-a6176fddf6fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:37.296455 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6b568522-f089-4275-acfb-a6176fddf6fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:37.296455 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1009/2017] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:33:36 2022] DELETE /identity/v3/projects/5cebcfd2bdde4adb91c68689dfc3e106 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:33:37.300714 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b568522-f089-4275-acfb-a6176fddf6fb admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:37.302574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6b568522-f089-4275-acfb-a6176fddf6fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:37.377252 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aed5a7-69ef-4717-862c-7da8b3af8849 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:37.384260 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd67a5-7082-4b7c-8131-e0a113bf583f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:37.388656 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd67a5-7082-4b7c-8131-e0a113bf583f admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:37.388656 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd67a5-7082-4b7c-8131-e0a113bf583f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:37.388656 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd67a5-7082-4b7c-8131-e0a113bf583f admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:37.388656 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fd67a5-7082-4b7c-8131-e0a113bf583f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:37.388985 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b568522-f089-4275-acfb-a6176fddf6fb admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:37.390941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fd67a5-7082-4b7c-8131-e0a113bf583f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:37.393869 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c1a5d-6e8d-4e39-be8f-023f02f121db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:37.395212 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c1a5d-6e8d-4e39-be8f-023f02f121db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:37.395498 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c1a5d-6e8d-4e39-be8f-023f02f121db admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:37.395774 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c1a5d-6e8d-4e39-be8f-023f02f121db admin admin] PATH_INFO: `/v3/users/9c1fef1e29bf4df587d101011850e281` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:37.397265 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c1a5d-6e8d-4e39-be8f-023f02f121db admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c1fef1e29bf4df587d101011850e281)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:37.399043 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c1a5d-6e8d-4e39-be8f-023f02f121db admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:37.542800 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1009/2018] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:33:37 2022] POST /identity/v3/users => generated 294 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:37.546525 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd67a5-7082-4b7c-8131-e0a113bf583f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:37.549047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-12fd67a5-7082-4b7c-8131-e0a113bf583f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:37.562455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd67a5-7082-4b7c-8131-e0a113bf583f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:37.567471 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b5ec5-e031-436a-a1ca-fc5f650b0c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:37.568296 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b5ec5-e031-436a-a1ca-fc5f650b0c05 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:37.568681 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b5ec5-e031-436a-a1ca-fc5f650b0c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:37.569456 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b5ec5-e031-436a-a1ca-fc5f650b0c05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:37.585703 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684b5ec5-e031-436a-a1ca-fc5f650b0c05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:37.585703 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684b5ec5-e031-436a-a1ca-fc5f650b0c05 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:37.602917 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1010/2019] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:33:37.604359 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b5ec5-e031-436a-a1ca-fc5f650b0c05 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:37.622004 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b5ec5-e031-436a-a1ca-fc5f650b0c05 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:37.627842 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5f89d-62e7-4062-ae12-811538d7ba23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:37.629019 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5f89d-62e7-4062-ae12-811538d7ba23 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:37.629310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5f89d-62e7-4062-ae12-811538d7ba23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:37.629588 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b5f89d-62e7-4062-ae12-811538d7ba23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:37.630244 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b5f89d-62e7-4062-ae12-811538d7ba23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:37.631950 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b5f89d-62e7-4062-ae12-811538d7ba23 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:37.656694 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1011/2020] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:37.660618 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5f89d-62e7-4062-ae12-811538d7ba23 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:37.674023 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b5f89d-62e7-4062-ae12-811538d7ba23 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:37.679729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5763b3-4f5f-447a-b10a-e4e51dc3285c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:37.680678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5763b3-4f5f-447a-b10a-e4e51dc3285c admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:37.680965 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5763b3-4f5f-447a-b10a-e4e51dc3285c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:37.681242 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5763b3-4f5f-447a-b10a-e4e51dc3285c admin admin] PATH_INFO: `/v3/projects/1a822457cfef430eb9869c1b0ed23e00/users/fb347c98e8af4f3fac1f33b64aac6142/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:37.696185 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5763b3-4f5f-447a-b10a-e4e51dc3285c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a822457cfef430eb9869c1b0ed23e00, user_id=fb347c98e8af4f3fac1f33b64aac6142, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:37.698871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5763b3-4f5f-447a-b10a-e4e51dc3285c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:37.744289 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e62c1a5d-6e8d-4e39-be8f-023f02f121db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c1fef1e29bf4df587d101011850e281'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:37.751290 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e62c1a5d-6e8d-4e39-be8f-023f02f121db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c1fef1e29bf4df587d101011850e281'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:37.831451 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1012/2021] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:37 2022] PUT /identity/v3/projects/1a822457cfef430eb9869c1b0ed23e00/users/fb347c98e8af4f3fac1f33b64aac6142/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:37.849317 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e62c1a5d-6e8d-4e39-be8f-023f02f121db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c1fef1e29bf4df587d101011850e281'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:37.849941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e62c1a5d-6e8d-4e39-be8f-023f02f121db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c1fef1e29bf4df587d101011850e281'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:37.856660 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab39ed-3ebb-4d82-b4db-bbcac6fb33ea None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:37.857199 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab39ed-3ebb-4d82-b4db-bbcac6fb33ea None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:37.857386 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab39ed-3ebb-4d82-b4db-bbcac6fb33ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:37.868386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e62c1a5d-6e8d-4e39-be8f-023f02f121db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c1fef1e29bf4df587d101011850e281'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:37.884796 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1010/2022] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:33:37 2022] DELETE /identity/v3/users/9c1fef1e29bf4df587d101011850e281 => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:37.910968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-73ab39ed-3ebb-4d82-b4db-bbcac6fb33ea None None] MFA Rules not processed for user `fb347c98e8af4f3fac1f33b64aac6142`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:37.912329 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-73ab39ed-3ebb-4d82-b4db-bbcac6fb33ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:38.010743 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1013/2023] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:38.015905 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c1a5d-6e8d-4e39-be8f-023f02f121db admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:38.017358 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e62c1a5d-6e8d-4e39-be8f-023f02f121db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:38.096397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c1a5d-6e8d-4e39-be8f-023f02f121db admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:38.096397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d08250-74fe-4a3b-a732-7330ca8ecac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:38.096397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d08250-74fe-4a3b-a732-7330ca8ecac1 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:38.096397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d08250-74fe-4a3b-a732-7330ca8ecac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:38.096826 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d08250-74fe-4a3b-a732-7330ca8ecac1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:38.096826 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d08250-74fe-4a3b-a732-7330ca8ecac1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:38.098384 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d08250-74fe-4a3b-a732-7330ca8ecac1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:38.227341 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1011/2024] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:33:38 2022] POST /identity/v3/projects => generated 393 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:38.231929 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab39ed-3ebb-4d82-b4db-bbcac6fb33ea None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:38.245853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab39ed-3ebb-4d82-b4db-bbcac6fb33ea None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:38.349353 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d4707-2458-4b5e-b076-2faef8812329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:38.349919 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d4707-2458-4b5e-b076-2faef8812329 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:38.350199 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d4707-2458-4b5e-b076-2faef8812329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:38.350471 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d4707-2458-4b5e-b076-2faef8812329 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:38.352043 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d4707-2458-4b5e-b076-2faef8812329 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:38.353722 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:38.359418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d4707-2458-4b5e-b076-2faef8812329 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:38.501177 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d08250-74fe-4a3b-a732-7330ca8ecac1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:38.503416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-87d08250-74fe-4a3b-a732-7330ca8ecac1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:38.516968 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d08250-74fe-4a3b-a732-7330ca8ecac1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:38.522293 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ea4d09-fd1b-4aa4-a418-fc88ae9e5591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:38.523142 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea4d09-fd1b-4aa4-a418-fc88ae9e5591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:38.523627 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea4d09-fd1b-4aa4-a418-fc88ae9e5591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:38.524090 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ea4d09-fd1b-4aa4-a418-fc88ae9e5591 admin admin] PATH_INFO: `/v3/projects/6aaf7e861ff6414fbf24e38bccaec480` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:38.525822 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea4d09-fd1b-4aa4-a418-fc88ae9e5591 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aaf7e861ff6414fbf24e38bccaec480)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:38.530537 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:38.531923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ea4d09-fd1b-4aa4-a418-fc88ae9e5591 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:38.537465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1014/2025] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:33:38 2022] POST /identity/v3/users => generated 303 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:33:38.541705 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d4707-2458-4b5e-b076-2faef8812329 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:38.556557 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d4707-2458-4b5e-b076-2faef8812329 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:38.566437 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38636ae-645b-46e4-a5bf-dbe7c8f54240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:38.566437 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38636ae-645b-46e4-a5bf-dbe7c8f54240 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:38.566437 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38636ae-645b-46e4-a5bf-dbe7c8f54240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:38.567000 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38636ae-645b-46e4-a5bf-dbe7c8f54240 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:38.567975 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38636ae-645b-46e4-a5bf-dbe7c8f54240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:38.570512 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38636ae-645b-46e4-a5bf-dbe7c8f54240 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:38.582749 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1015/2026] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:38.586215 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38636ae-645b-46e4-a5bf-dbe7c8f54240 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:38.597918 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38636ae-645b-46e4-a5bf-dbe7c8f54240 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:38.603461 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8858a1-2e63-4c3a-bc64-3baaa54c4a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:38.604060 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8858a1-2e63-4c3a-bc64-3baaa54c4a1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:38.604337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8858a1-2e63-4c3a-bc64-3baaa54c4a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:38.604609 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8858a1-2e63-4c3a-bc64-3baaa54c4a1f admin admin] PATH_INFO: `/v3/projects/d8605b7ac11c402680ec5bd1709cadec/users/591ca22cdf194d969dd85604ffb84c6b/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:38.621030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8858a1-2e63-4c3a-bc64-3baaa54c4a1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8605b7ac11c402680ec5bd1709cadec, user_id=591ca22cdf194d969dd85604ffb84c6b, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:38.625276 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8858a1-2e63-4c3a-bc64-3baaa54c4a1f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:38.716443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1016/2027] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:38 2022] PUT /identity/v3/projects/d8605b7ac11c402680ec5bd1709cadec/users/591ca22cdf194d969dd85604ffb84c6b/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:33:38.720463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8858a1-2e63-4c3a-bc64-3baaa54c4a1f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:38.722931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-32ea4d09-fd1b-4aa4-a418-fc88ae9e5591 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aaf7e861ff6414fbf24e38bccaec480', 'request_id': 'req-32ea4d09-fd1b-4aa4-a418-fc88ae9e5591'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:38.791038 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-32ea4d09-fd1b-4aa4-a418-fc88ae9e5591 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aaf7e861ff6414fbf24e38bccaec480', 'request_id': 'req-32ea4d09-fd1b-4aa4-a418-fc88ae9e5591'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:38.797184 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8858a1-2e63-4c3a-bc64-3baaa54c4a1f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:38.803218 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24f244-be7f-485c-ab0b-3013e5def011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:38.804489 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24f244-be7f-485c-ab0b-3013e5def011 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:38.804902 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24f244-be7f-485c-ab0b-3013e5def011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:38.805280 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b24f244-be7f-485c-ab0b-3013e5def011 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:38.806046 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b24f244-be7f-485c-ab0b-3013e5def011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:38.806277 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-32ea4d09-fd1b-4aa4-a418-fc88ae9e5591 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aaf7e861ff6414fbf24e38bccaec480', 'request_id': 'req-32ea4d09-fd1b-4aa4-a418-fc88ae9e5591'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:38.808307 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b24f244-be7f-485c-ab0b-3013e5def011 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:38.823065 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1017/2028] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:38.834777 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-32ea4d09-fd1b-4aa4-a418-fc88ae9e5591 admin admin] The token cache is being invalidate because project 6aaf7e861ff6414fbf24e38bccaec480 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:33:38.834777 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-32ea4d09-fd1b-4aa4-a418-fc88ae9e5591 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:38.834777 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-32ea4d09-fd1b-4aa4-a418-fc88ae9e5591 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:38.834777 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1012/2029] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:33:38 2022] DELETE /identity/v3/projects/6aaf7e861ff6414fbf24e38bccaec480 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:33:38.836600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24f244-be7f-485c-ab0b-3013e5def011 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:38.856341 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b732f-4ac0-4d48-9504-756f278ba4f5 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:38.856623 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b732f-4ac0-4d48-9504-756f278ba4f5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:38.857192 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b732f-4ac0-4d48-9504-756f278ba4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:38.863382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2b24f244-be7f-485c-ab0b-3013e5def011 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:38.898102 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b24f244-be7f-485c-ab0b-3013e5def011 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:38.906760 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87d86d-2fae-4f79-b222-66df448a65f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:38.907691 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87d86d-2fae-4f79-b222-66df448a65f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:38.907973 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87d86d-2fae-4f79-b222-66df448a65f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:38.908254 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87d86d-2fae-4f79-b222-66df448a65f1 admin admin] PATH_INFO: `/v3/projects/d8605b7ac11c402680ec5bd1709cadec/users/591ca22cdf194d969dd85604ffb84c6b/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:38.910376 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc87d86d-2fae-4f79-b222-66df448a65f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8605b7ac11c402680ec5bd1709cadec, user_id=591ca22cdf194d969dd85604ffb84c6b, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:38.915259 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc87d86d-2fae-4f79-b222-66df448a65f1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:38.915435 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-879b732f-4ac0-4d48-9504-756f278ba4f5 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:38.915435 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-879b732f-4ac0-4d48-9504-756f278ba4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:38.928901 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1018/2030] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:38 2022] PUT /identity/v3/projects/d8605b7ac11c402680ec5bd1709cadec/users/591ca22cdf194d969dd85604ffb84c6b/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:38.944248 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be990d-771c-420c-a665-e55b0b401d05 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:38.944248 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be990d-771c-420c-a665-e55b0b401d05 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:38.944248 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be990d-771c-420c-a665-e55b0b401d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:38.978802 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-80be990d-771c-420c-a665-e55b0b401d05 None None] MFA Rules not processed for user `591ca22cdf194d969dd85604ffb84c6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:38.980247 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-80be990d-771c-420c-a665-e55b0b401d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:39.011400 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1013/2031] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:39.024671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2ae29-59fc-4a8a-808c-0bbc128a8fee None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:39.024962 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2ae29-59fc-4a8a-808c-0bbc128a8fee None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:39.025343 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2ae29-59fc-4a8a-808c-0bbc128a8fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:39.060505 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f5b2ae29-59fc-4a8a-808c-0bbc128a8fee None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:39.061886 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f5b2ae29-59fc-4a8a-808c-0bbc128a8fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:39.072736 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1014/2032] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:39.087185 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2ae29-59fc-4a8a-808c-0bbc128a8fee None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:39.092447 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1019/2033] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:38 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:33:39.099380 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2ae29-59fc-4a8a-808c-0bbc128a8fee None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:39.104718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7662b6-e8ae-4909-bc36-04a6983ef789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:39.105371 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7662b6-e8ae-4909-bc36-04a6983ef789 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:39.105746 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7662b6-e8ae-4909-bc36-04a6983ef789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:39.106089 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7662b6-e8ae-4909-bc36-04a6983ef789 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:39.106880 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7662b6-e8ae-4909-bc36-04a6983ef789 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:33:39.107377 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7662b6-e8ae-4909-bc36-04a6983ef789 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:39.107879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be990d-771c-420c-a665-e55b0b401d05 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:39.109119 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7662b6-e8ae-4909-bc36-04a6983ef789 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:39.109804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-80be990d-771c-420c-a665-e55b0b401d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:39.121693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1015/2034] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:33:39.126194 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7662b6-e8ae-4909-bc36-04a6983ef789 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:39.138532 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7662b6-e8ae-4909-bc36-04a6983ef789 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:39.143963 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df661358-e521-4b30-9342-d582ca6bc4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:39.144788 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df661358-e521-4b30-9342-d582ca6bc4ce admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:39.145228 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df661358-e521-4b30-9342-d582ca6bc4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:39.145687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df661358-e521-4b30-9342-d582ca6bc4ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:39.146327 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df661358-e521-4b30-9342-d582ca6bc4ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:39.148839 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df661358-e521-4b30-9342-d582ca6bc4ce admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:39.180079 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be990d-771c-420c-a665-e55b0b401d05 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:39.183652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06a94c-3f1a-46db-a237-a7b31f81382b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:39.184287 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06a94c-3f1a-46db-a237-a7b31f81382b service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:39.184566 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06a94c-3f1a-46db-a237-a7b31f81382b service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:39.184875 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06a94c-3f1a-46db-a237-a7b31f81382b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:39.193917 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb06a94c-3f1a-46db-a237-a7b31f81382b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:39.195947 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb06a94c-3f1a-46db-a237-a7b31f81382b service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:39.206171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1020/2035] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:39 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:33:39.263463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06a94c-3f1a-46db-a237-a7b31f81382b service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:39.273805 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06a94c-3f1a-46db-a237-a7b31f81382b service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:39.287256 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a44fd7-85a8-4667-b147-dffe039143a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:39.287256 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a44fd7-85a8-4667-b147-dffe039143a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:39.287256 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a44fd7-85a8-4667-b147-dffe039143a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:39.287256 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a44fd7-85a8-4667-b147-dffe039143a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:39.287613 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d3a44fd7-85a8-4667-b147-dffe039143a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:39.292895 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1016/2036] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:33:39 2022] POST /identity/v3/projects => generated 381 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:33:39.297222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df661358-e521-4b30-9342-d582ca6bc4ce admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:39.372892 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a44fd7-85a8-4667-b147-dffe039143a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:39.374493 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a44fd7-85a8-4667-b147-dffe039143a5 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:39.380639 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df661358-e521-4b30-9342-d582ca6bc4ce admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:39.385675 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1021/2037] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:39 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:33:39.389972 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266eb03b-2377-4f0f-9ef9-c63e9688ceb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:39.389972 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266eb03b-2377-4f0f-9ef9-c63e9688ceb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:39.389972 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266eb03b-2377-4f0f-9ef9-c63e9688ceb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:39.389972 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266eb03b-2377-4f0f-9ef9-c63e9688ceb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:39.390334 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266eb03b-2377-4f0f-9ef9-c63e9688ceb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:39.390334 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266eb03b-2377-4f0f-9ef9-c63e9688ceb8 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:39.556639 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1017/2038] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:33:39 2022] POST /identity/v3/users => generated 291 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:33:39.557989 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a44fd7-85a8-4667-b147-dffe039143a5 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:39.568952 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a44fd7-85a8-4667-b147-dffe039143a5 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:39.574374 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59acfb7d-3971-4715-9ebe-287112066397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:39.581944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59acfb7d-3971-4715-9ebe-287112066397 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:39.581944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59acfb7d-3971-4715-9ebe-287112066397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:39.581944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59acfb7d-3971-4715-9ebe-287112066397 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:39.581944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59acfb7d-3971-4715-9ebe-287112066397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:39.587413 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59acfb7d-3971-4715-9ebe-287112066397 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:39.598298 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1022/2039] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:33:39.602578 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266eb03b-2377-4f0f-9ef9-c63e9688ceb8 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:39.612855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266eb03b-2377-4f0f-9ef9-c63e9688ceb8 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:39.618289 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf4de1-4889-422f-a665-cf6235d8a4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:39.618940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf4de1-4889-422f-a665-cf6235d8a4e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:39.619255 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf4de1-4889-422f-a665-cf6235d8a4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:39.619475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf4de1-4889-422f-a665-cf6235d8a4e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:39.620115 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf4de1-4889-422f-a665-cf6235d8a4e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:39.621699 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf4de1-4889-422f-a665-cf6235d8a4e1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:39.633943 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1018/2040] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:39.639157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59acfb7d-3971-4715-9ebe-287112066397 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:39.655062 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59acfb7d-3971-4715-9ebe-287112066397 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:39.666973 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29192525-40c9-4eca-bff2-be788ddb8554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:39.667665 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29192525-40c9-4eca-bff2-be788ddb8554 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:39.668004 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29192525-40c9-4eca-bff2-be788ddb8554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:39.668667 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29192525-40c9-4eca-bff2-be788ddb8554 admin admin] PATH_INFO: `/v3/projects/f9c3aa3e6dbc40568c6184e59ccfd841/users/efcfb3ef7f3745c3a5fe51827eb32201/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:39.688698 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29192525-40c9-4eca-bff2-be788ddb8554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9c3aa3e6dbc40568c6184e59ccfd841, user_id=efcfb3ef7f3745c3a5fe51827eb32201, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:39.691404 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29192525-40c9-4eca-bff2-be788ddb8554 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:39.774391 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1023/2041] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:39 2022] PUT /identity/v3/projects/f9c3aa3e6dbc40568c6184e59ccfd841/users/efcfb3ef7f3745c3a5fe51827eb32201/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:33:39.784533 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf6639-bf71-4bc4-906b-3795d27fa118 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:39.784822 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf6639-bf71-4bc4-906b-3795d27fa118 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:39.785095 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf6639-bf71-4bc4-906b-3795d27fa118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:39.834010 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1dbf6639-bf71-4bc4-906b-3795d27fa118 None None] MFA Rules not processed for user `efcfb3ef7f3745c3a5fe51827eb32201`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:39.835351 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1dbf6639-bf71-4bc4-906b-3795d27fa118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:39.951392 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1019/2042] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:40.042293 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29192525-40c9-4eca-bff2-be788ddb8554 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:40.110300 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29192525-40c9-4eca-bff2-be788ddb8554 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:40.115039 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efdd7ce-1b52-4819-bd94-f0643374759b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:40.115980 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdd7ce-1b52-4819-bd94-f0643374759b service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:40.118001 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdd7ce-1b52-4819-bd94-f0643374759b service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:40.118419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdd7ce-1b52-4819-bd94-f0643374759b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:40.132260 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efdd7ce-1b52-4819-bd94-f0643374759b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:40.132429 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efdd7ce-1b52-4819-bd94-f0643374759b service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:40.144765 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1024/2043] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:40 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:33:40.170246 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf6639-bf71-4bc4-906b-3795d27fa118 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:40.255800 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf6639-bf71-4bc4-906b-3795d27fa118 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:40.261271 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8876a5d4-a376-4f16-aca8-5ae355e3a1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:40.267750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876a5d4-a376-4f16-aca8-5ae355e3a1ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:40.272036 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876a5d4-a376-4f16-aca8-5ae355e3a1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:40.272036 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8876a5d4-a376-4f16-aca8-5ae355e3a1ac admin admin] PATH_INFO: `/v3/users/efcfb3ef7f3745c3a5fe51827eb32201` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:40.272036 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876a5d4-a376-4f16-aca8-5ae355e3a1ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=efcfb3ef7f3745c3a5fe51827eb32201)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:40.272036 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8876a5d4-a376-4f16-aca8-5ae355e3a1ac admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:40.535963 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8876a5d4-a376-4f16-aca8-5ae355e3a1ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efcfb3ef7f3745c3a5fe51827eb32201'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:40.544672 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8876a5d4-a376-4f16-aca8-5ae355e3a1ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efcfb3ef7f3745c3a5fe51827eb32201'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:40.624895 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efdd7ce-1b52-4819-bd94-f0643374759b service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:40.626499 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0efdd7ce-1b52-4819-bd94-f0643374759b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:40.641294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efdd7ce-1b52-4819-bd94-f0643374759b service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:40.646303 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c6c4b-85ed-435c-a55c-f3f72ac8ec6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:40.647077 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c6c4b-85ed-435c-a55c-f3f72ac8ec6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:40.648144 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c6c4b-85ed-435c-a55c-f3f72ac8ec6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:40.648521 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c6c4b-85ed-435c-a55c-f3f72ac8ec6f admin admin] PATH_INFO: `/v3/users/fe7a2ddb181647f395e4b6e1103daf96` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:40.649750 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c6c4b-85ed-435c-a55c-f3f72ac8ec6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe7a2ddb181647f395e4b6e1103daf96)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:40.651706 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c6c4b-85ed-435c-a55c-f3f72ac8ec6f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:40.661579 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8876a5d4-a376-4f16-aca8-5ae355e3a1ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efcfb3ef7f3745c3a5fe51827eb32201'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:40.667431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8876a5d4-a376-4f16-aca8-5ae355e3a1ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efcfb3ef7f3745c3a5fe51827eb32201'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:40.677049 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8876a5d4-a376-4f16-aca8-5ae355e3a1ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efcfb3ef7f3745c3a5fe51827eb32201'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:40.685851 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1020/2044] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:33:40 2022] DELETE /identity/v3/users/efcfb3ef7f3745c3a5fe51827eb32201 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:40.689972 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e6a80-d025-4ede-b07a-a49f6e1bcf86 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:40.690477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e6a80-d025-4ede-b07a-a49f6e1bcf86 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:40.696265 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:40.696265 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e6a80-d025-4ede-b07a-a49f6e1bcf86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:40.712319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a53e6a80-d025-4ede-b07a-a49f6e1bcf86 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:40.713613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a53e6a80-d025-4ede-b07a-a49f6e1bcf86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:40.773687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1021/2045] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:33:40.783231 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e6a80-d025-4ede-b07a-a49f6e1bcf86 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:40.784544 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a53e6a80-d025-4ede-b07a-a49f6e1bcf86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:40.843227 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e6a80-d025-4ede-b07a-a49f6e1bcf86 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:40.846420 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-992c6c4b-85ed-435c-a55c-f3f72ac8ec6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe7a2ddb181647f395e4b6e1103daf96'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:40.848056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b550a62-7c9f-4fae-b0eb-20439d98a0c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:40.848747 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b550a62-7c9f-4fae-b0eb-20439d98a0c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:40.849163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b550a62-7c9f-4fae-b0eb-20439d98a0c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:40.849445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b550a62-7c9f-4fae-b0eb-20439d98a0c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:40.859138 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b550a62-7c9f-4fae-b0eb-20439d98a0c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:40.860707 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b550a62-7c9f-4fae-b0eb-20439d98a0c6 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:40.870145 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1022/2046] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:33:40.935419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-992c6c4b-85ed-435c-a55c-f3f72ac8ec6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe7a2ddb181647f395e4b6e1103daf96'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:40.940248 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-992c6c4b-85ed-435c-a55c-f3f72ac8ec6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe7a2ddb181647f395e4b6e1103daf96'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:40.940248 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-992c6c4b-85ed-435c-a55c-f3f72ac8ec6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe7a2ddb181647f395e4b6e1103daf96'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:40.950830 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-992c6c4b-85ed-435c-a55c-f3f72ac8ec6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe7a2ddb181647f395e4b6e1103daf96'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:40.964191 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1025/2047] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:33:40 2022] DELETE /identity/v3/users/fe7a2ddb181647f395e4b6e1103daf96 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:40.968689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abd6bd-f997-4c7c-b476-735bc9eed26c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:40.968995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abd6bd-f997-4c7c-b476-735bc9eed26c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:40.969255 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abd6bd-f997-4c7c-b476-735bc9eed26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:40.987332 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:40.990873 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-37abd6bd-f997-4c7c-b476-735bc9eed26c None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:40.990873 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-37abd6bd-f997-4c7c-b476-735bc9eed26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:41.056113 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1023/2048] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:41.065761 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c6c4b-85ed-435c-a55c-f3f72ac8ec6f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:41.067239 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-992c6c4b-85ed-435c-a55c-f3f72ac8ec6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:41.159941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c6c4b-85ed-435c-a55c-f3f72ac8ec6f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:41.164975 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1892806-5793-4ab9-9f19-13c2d8a46db9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:41.166205 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1892806-5793-4ab9-9f19-13c2d8a46db9 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:41.167101 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1892806-5793-4ab9-9f19-13c2d8a46db9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:41.168167 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1892806-5793-4ab9-9f19-13c2d8a46db9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:41.177906 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1892806-5793-4ab9-9f19-13c2d8a46db9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:41.186926 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1892806-5793-4ab9-9f19-13c2d8a46db9 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:41.198224 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1026/2049] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:33:41.393573 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37abd6bd-f997-4c7c-b476-735bc9eed26c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:41.395067 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-37abd6bd-f997-4c7c-b476-735bc9eed26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:41.406900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37abd6bd-f997-4c7c-b476-735bc9eed26c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:41.414116 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e35cdad-f3ba-4941-88cc-1dbc8f77a1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:41.414116 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35cdad-f3ba-4941-88cc-1dbc8f77a1ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:41.414116 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35cdad-f3ba-4941-88cc-1dbc8f77a1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:41.414116 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35cdad-f3ba-4941-88cc-1dbc8f77a1ac admin admin] PATH_INFO: `/v3/projects/f9c3aa3e6dbc40568c6184e59ccfd841` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:41.414539 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e35cdad-f3ba-4941-88cc-1dbc8f77a1ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9c3aa3e6dbc40568c6184e59ccfd841)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:41.416439 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e35cdad-f3ba-4941-88cc-1dbc8f77a1ac admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:41.541208 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9e35cdad-f3ba-4941-88cc-1dbc8f77a1ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9c3aa3e6dbc40568c6184e59ccfd841', 'request_id': 'req-9e35cdad-f3ba-4941-88cc-1dbc8f77a1ac'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:41.625318 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9e35cdad-f3ba-4941-88cc-1dbc8f77a1ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9c3aa3e6dbc40568c6184e59ccfd841', 'request_id': 'req-9e35cdad-f3ba-4941-88cc-1dbc8f77a1ac'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:41.633157 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9e35cdad-f3ba-4941-88cc-1dbc8f77a1ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9c3aa3e6dbc40568c6184e59ccfd841', 'request_id': 'req-9e35cdad-f3ba-4941-88cc-1dbc8f77a1ac'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:41.654481 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9e35cdad-f3ba-4941-88cc-1dbc8f77a1ac admin admin] The token cache is being invalidate because project f9c3aa3e6dbc40568c6184e59ccfd841 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:33:41.655803 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9e35cdad-f3ba-4941-88cc-1dbc8f77a1ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:41.655803 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9e35cdad-f3ba-4941-88cc-1dbc8f77a1ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:41.656973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1024/2050] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:33:41 2022] DELETE /identity/v3/projects/f9c3aa3e6dbc40568c6184e59ccfd841 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:41.662043 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc52f0d7-0296-4fb5-8073-5dea54cef3c5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:41.662335 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc52f0d7-0296-4fb5-8073-5dea54cef3c5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:41.662628 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc52f0d7-0296-4fb5-8073-5dea54cef3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:41.693515 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-bc52f0d7-0296-4fb5-8073-5dea54cef3c5 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:41.694881 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bc52f0d7-0296-4fb5-8073-5dea54cef3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:41.757802 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1027/2051] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:41.766996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37681475-a704-479b-b645-777bc3137f60 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:41.767430 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37681475-a704-479b-b645-777bc3137f60 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:41.767729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37681475-a704-479b-b645-777bc3137f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:41.792003 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-37681475-a704-479b-b645-777bc3137f60 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:41.793248 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-37681475-a704-479b-b645-777bc3137f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:41.803317 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1025/2052] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:41.809968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc52f0d7-0296-4fb5-8073-5dea54cef3c5 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:41.820420 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc52f0d7-0296-4fb5-8073-5dea54cef3c5 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:41.825239 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eee75f-adfb-491d-9eaa-23ad8033fc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:41.830133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eee75f-adfb-491d-9eaa-23ad8033fc35 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:41.830133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eee75f-adfb-491d-9eaa-23ad8033fc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:41.830133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eee75f-adfb-491d-9eaa-23ad8033fc35 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:41.830133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eee75f-adfb-491d-9eaa-23ad8033fc35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:33:41.830133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eee75f-adfb-491d-9eaa-23ad8033fc35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:41.830133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eee75f-adfb-491d-9eaa-23ad8033fc35 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:41.836478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37681475-a704-479b-b645-777bc3137f60 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:41.839064 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-37681475-a704-479b-b645-777bc3137f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:41.856363 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1028/2053] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:33:41.860928 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37681475-a704-479b-b645-777bc3137f60 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:41.862468 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eee75f-adfb-491d-9eaa-23ad8033fc35 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:41.866043 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db71dcce-e0aa-4332-a2c9-b33411056673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:41.866924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71dcce-e0aa-4332-a2c9-b33411056673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:41.867289 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71dcce-e0aa-4332-a2c9-b33411056673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:41.867615 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71dcce-e0aa-4332-a2c9-b33411056673 admin admin] PATH_INFO: `/v3/projects/1c7edb1b29324fa791e7ea3bccb68284` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:41.868991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db71dcce-e0aa-4332-a2c9-b33411056673 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c7edb1b29324fa791e7ea3bccb68284)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:41.871611 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db71dcce-e0aa-4332-a2c9-b33411056673 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:41.874640 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eee75f-adfb-491d-9eaa-23ad8033fc35 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:41.885038 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1334a87-d5e2-4f4a-b33c-694541f24900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:41.892017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1334a87-d5e2-4f4a-b33c-694541f24900 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:41.892017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1334a87-d5e2-4f4a-b33c-694541f24900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:41.892017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1334a87-d5e2-4f4a-b33c-694541f24900 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:41.892017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1334a87-d5e2-4f4a-b33c-694541f24900 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:41.892017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1334a87-d5e2-4f4a-b33c-694541f24900 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:41.980753 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1029/2054] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:33:41 2022] POST /identity/v3/projects => generated 369 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:33:41.981842 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1334a87-d5e2-4f4a-b33c-694541f24900 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:41.987925 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-db71dcce-e0aa-4332-a2c9-b33411056673 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c7edb1b29324fa791e7ea3bccb68284', 'request_id': 'req-db71dcce-e0aa-4332-a2c9-b33411056673'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:42.044157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1334a87-d5e2-4f4a-b33c-694541f24900 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:42.048812 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9496750f-2964-4e5a-a0d8-dc8add1cbe7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:42.049407 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496750f-2964-4e5a-a0d8-dc8add1cbe7a admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:42.049727 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496750f-2964-4e5a-a0d8-dc8add1cbe7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:42.049970 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496750f-2964-4e5a-a0d8-dc8add1cbe7a admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:42.050619 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9496750f-2964-4e5a-a0d8-dc8add1cbe7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:42.052564 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9496750f-2964-4e5a-a0d8-dc8add1cbe7a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:42.064562 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-db71dcce-e0aa-4332-a2c9-b33411056673 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c7edb1b29324fa791e7ea3bccb68284', 'request_id': 'req-db71dcce-e0aa-4332-a2c9-b33411056673'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:42.072414 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-db71dcce-e0aa-4332-a2c9-b33411056673 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c7edb1b29324fa791e7ea3bccb68284', 'request_id': 'req-db71dcce-e0aa-4332-a2c9-b33411056673'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:42.084212 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-db71dcce-e0aa-4332-a2c9-b33411056673 admin admin] The token cache is being invalidate because project 1c7edb1b29324fa791e7ea3bccb68284 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:33:42.084509 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-db71dcce-e0aa-4332-a2c9-b33411056673 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:42.085799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-db71dcce-e0aa-4332-a2c9-b33411056673 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:42.087316 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1026/2055] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:33:41 2022] DELETE /identity/v3/projects/1c7edb1b29324fa791e7ea3bccb68284 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:42.091998 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cf744-6f19-4eb8-a5dc-b6045f61c07a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:42.092278 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cf744-6f19-4eb8-a5dc-b6045f61c07a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:42.092543 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cf744-6f19-4eb8-a5dc-b6045f61c07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:42.111469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6d2cf744-6f19-4eb8-a5dc-b6045f61c07a None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:42.112659 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6d2cf744-6f19-4eb8-a5dc-b6045f61c07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:42.121015 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1027/2056] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:33:42.134428 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d319338-cc88-4989-a320-b7c373addd51 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:42.134729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d319338-cc88-4989-a320-b7c373addd51 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:42.135021 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d319338-cc88-4989-a320-b7c373addd51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:42.154118 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1030/2057] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:33:41 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:42.162956 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0d319338-cc88-4989-a320-b7c373addd51 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:42.162956 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0d319338-cc88-4989-a320-b7c373addd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:42.165727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1028/2058] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:33:42.165843 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9496750f-2964-4e5a-a0d8-dc8add1cbe7a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:42.167541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9496750f-2964-4e5a-a0d8-dc8add1cbe7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:42.169705 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d319338-cc88-4989-a320-b7c373addd51 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:42.178961 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d319338-cc88-4989-a320-b7c373addd51 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:42.184557 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c0480-0d4c-4a88-a929-08893b613f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:42.185167 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c0480-0d4c-4a88-a929-08893b613f65 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:42.185445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c0480-0d4c-4a88-a929-08893b613f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:42.185721 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c0480-0d4c-4a88-a929-08893b613f65 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:42.186367 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c0480-0d4c-4a88-a929-08893b613f65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:33:42.186546 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9496750f-2964-4e5a-a0d8-dc8add1cbe7a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:42.186730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c0480-0d4c-4a88-a929-08893b613f65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:42.188310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c0480-0d4c-4a88-a929-08893b613f65 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:42.191642 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab279f-67fe-47dc-a91d-3579d04de106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:42.192030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab279f-67fe-47dc-a91d-3579d04de106 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:42.192409 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab279f-67fe-47dc-a91d-3579d04de106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:42.192728 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab279f-67fe-47dc-a91d-3579d04de106 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:42.193329 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab279f-67fe-47dc-a91d-3579d04de106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:42.195449 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab279f-67fe-47dc-a91d-3579d04de106 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:42.210506 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1031/2059] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:33:42.211419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab279f-67fe-47dc-a91d-3579d04de106 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:42.212005 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1029/2060] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:33:42.212821 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c0480-0d4c-4a88-a929-08893b613f65 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:42.226185 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab279f-67fe-47dc-a91d-3579d04de106 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:42.226312 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c0480-0d4c-4a88-a929-08893b613f65 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:42.226459 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e58c1-454a-4049-bbb6-cbd7e374944b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:42.227067 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e58c1-454a-4049-bbb6-cbd7e374944b admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:42.227348 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e58c1-454a-4049-bbb6-cbd7e374944b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:42.227618 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e58c1-454a-4049-bbb6-cbd7e374944b admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:42.228245 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8e58c1-454a-4049-bbb6-cbd7e374944b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:42.230140 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8e58c1-454a-4049-bbb6-cbd7e374944b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:42.230322 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a86c4b-aaed-4f58-9262-d8829cc4b776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:42.231172 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a86c4b-aaed-4f58-9262-d8829cc4b776 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:42.231492 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a86c4b-aaed-4f58-9262-d8829cc4b776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:42.231862 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a86c4b-aaed-4f58-9262-d8829cc4b776 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:42.232618 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a86c4b-aaed-4f58-9262-d8829cc4b776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:42.234272 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a86c4b-aaed-4f58-9262-d8829cc4b776 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:42.245545 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1030/2061] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:42.249750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a86c4b-aaed-4f58-9262-d8829cc4b776 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:42.260360 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a86c4b-aaed-4f58-9262-d8829cc4b776 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:42.265166 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd263e6-f493-416e-aa03-358d20b31ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:42.265815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd263e6-f493-416e-aa03-358d20b31ab2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:42.266092 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd263e6-f493-416e-aa03-358d20b31ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:42.266361 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd263e6-f493-416e-aa03-358d20b31ab2 admin admin] PATH_INFO: `/v3/projects/d380d80d37b943f695988ebc32eb39a3/users/d21a777bb5264f1990e3cc2a73837b99/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:42.281338 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd263e6-f493-416e-aa03-358d20b31ab2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d380d80d37b943f695988ebc32eb39a3, user_id=d21a777bb5264f1990e3cc2a73837b99, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:42.283984 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd263e6-f493-416e-aa03-358d20b31ab2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:42.385106 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1032/2062] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:33:42 2022] POST /identity/v3/projects => generated 383 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:42.389396 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e58c1-454a-4049-bbb6-cbd7e374944b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:42.405451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1031/2063] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:42 2022] PUT /identity/v3/projects/d380d80d37b943f695988ebc32eb39a3/users/d21a777bb5264f1990e3cc2a73837b99/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:42.415669 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df39742-0369-472d-9d4b-237b0d5b30e8 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:42.416182 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df39742-0369-472d-9d4b-237b0d5b30e8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:42.416763 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df39742-0369-472d-9d4b-237b0d5b30e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:42.460627 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-9df39742-0369-472d-9d4b-237b0d5b30e8 None None] MFA Rules not processed for user `d21a777bb5264f1990e3cc2a73837b99`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:42.461878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9df39742-0369-472d-9d4b-237b0d5b30e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:42.473827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e58c1-454a-4049-bbb6-cbd7e374944b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:42.528224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1032/2064] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:42.529104 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1293ea-08ee-4c11-a9fc-b1186e33da6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:42.529726 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1293ea-08ee-4c11-a9fc-b1186e33da6e admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:42.530097 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1293ea-08ee-4c11-a9fc-b1186e33da6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:42.530455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1293ea-08ee-4c11-a9fc-b1186e33da6e admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:42.531260 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1293ea-08ee-4c11-a9fc-b1186e33da6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:42.533744 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1293ea-08ee-4c11-a9fc-b1186e33da6e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:42.539752 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df39742-0369-472d-9d4b-237b0d5b30e8 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:42.541163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9df39742-0369-472d-9d4b-237b0d5b30e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:42.632496 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1033/2065] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:33:42 2022] POST /identity/v3/users => generated 292 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:33:42.635485 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1293ea-08ee-4c11-a9fc-b1186e33da6e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:42.638476 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df39742-0369-472d-9d4b-237b0d5b30e8 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:42.644428 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea919b8-2f80-41e9-b424-d5215d4caed7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b555bb94ce44ce918cbf2a012576a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:42.645111 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea919b8-2f80-41e9-b424-d5215d4caed7 service glance] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:42.645393 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea919b8-2f80-41e9-b424-d5215d4caed7 service glance] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:42.645669 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea919b8-2f80-41e9-b424-d5215d4caed7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:42.647528 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1293ea-08ee-4c11-a9fc-b1186e33da6e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:42.652469 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ea773-3e16-45a2-b8bc-7f69f0741fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:42.653285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ea773-3e16-45a2-b8bc-7f69f0741fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:42.653631 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ea773-3e16-45a2-b8bc-7f69f0741fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:42.653987 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ea773-3e16-45a2-b8bc-7f69f0741fa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:42.657367 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ea773-3e16-45a2-b8bc-7f69f0741fa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:42.657500 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea919b8-2f80-41e9-b424-d5215d4caed7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:42.657500 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea919b8-2f80-41e9-b424-d5215d4caed7 service glance] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:42.657667 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ea773-3e16-45a2-b8bc-7f69f0741fa8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:42.666533 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1034/2066] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:33:42.667222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1033/2067] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:42 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:33:42.670643 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ea773-3e16-45a2-b8bc-7f69f0741fa8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:42.681144 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ea773-3e16-45a2-b8bc-7f69f0741fa8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:42.689853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7730f-247a-4d6b-a0a6-6bab14efa626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:42.690477 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7730f-247a-4d6b-a0a6-6bab14efa626 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:42.690795 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7730f-247a-4d6b-a0a6-6bab14efa626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:42.691168 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7730f-247a-4d6b-a0a6-6bab14efa626 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:42.691822 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d7730f-247a-4d6b-a0a6-6bab14efa626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:42.693634 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d7730f-247a-4d6b-a0a6-6bab14efa626 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:42.705837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1035/2068] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:33:42.712598 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea919b8-2f80-41e9-b424-d5215d4caed7 service glance] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:42.723831 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea919b8-2f80-41e9-b424-d5215d4caed7 service glance] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:42.728527 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28a571-26fd-4a52-a9ef-92701e92af2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:42.729367 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28a571-26fd-4a52-a9ef-92701e92af2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:42.729885 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28a571-26fd-4a52-a9ef-92701e92af2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:42.730169 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28a571-26fd-4a52-a9ef-92701e92af2b admin admin] PATH_INFO: `/v3/projects/28efe27ecdf04c4581e855ddd7f07bc5/users/ceb389101cdc4f8db5a69d0c653fc15e/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:42.745717 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb28a571-26fd-4a52-a9ef-92701e92af2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=28efe27ecdf04c4581e855ddd7f07bc5, user_id=ceb389101cdc4f8db5a69d0c653fc15e, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:42.748388 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb28a571-26fd-4a52-a9ef-92701e92af2b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:42.826498 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1034/2069] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:42 2022] PUT /identity/v3/projects/28efe27ecdf04c4581e855ddd7f07bc5/users/ceb389101cdc4f8db5a69d0c653fc15e/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:42.837715 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3a47e-6b1a-4e1e-a7ab-708834d8e7a3 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:42.838000 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3a47e-6b1a-4e1e-a7ab-708834d8e7a3 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:42.838273 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3a47e-6b1a-4e1e-a7ab-708834d8e7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:42.868071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-aae3a47e-6b1a-4e1e-a7ab-708834d8e7a3 None None] MFA Rules not processed for user `ceb389101cdc4f8db5a69d0c653fc15e`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:42.869327 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-aae3a47e-6b1a-4e1e-a7ab-708834d8e7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:42.947281 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1036/2070] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:42.948661 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28a571-26fd-4a52-a9ef-92701e92af2b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:43.014746 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28a571-26fd-4a52-a9ef-92701e92af2b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:43.019958 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f72f49-e794-43d8-8a85-916ac622b95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:43.020544 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f72f49-e794-43d8-8a85-916ac622b95c admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:43.020830 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f72f49-e794-43d8-8a85-916ac622b95c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:43.021235 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f72f49-e794-43d8-8a85-916ac622b95c admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:43.021922 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f72f49-e794-43d8-8a85-916ac622b95c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:43.023902 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f72f49-e794-43d8-8a85-916ac622b95c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:43.090656 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1035/2071] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:33:42 2022] POST /identity/v3/projects => generated 385 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:43.095021 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3a47e-6b1a-4e1e-a7ab-708834d8e7a3 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:43.106409 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3a47e-6b1a-4e1e-a7ab-708834d8e7a3 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:43.160453 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de11e8-7eb9-4217-8580-39e8cd4596d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:43.161205 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de11e8-7eb9-4217-8580-39e8cd4596d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:43.161511 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de11e8-7eb9-4217-8580-39e8cd4596d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:43.161804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de11e8-7eb9-4217-8580-39e8cd4596d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:43.162426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42de11e8-7eb9-4217-8580-39e8cd4596d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:43.165036 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42de11e8-7eb9-4217-8580-39e8cd4596d6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:43.276927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:43.286188 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1037/2072] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:33:43 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:33:43.287399 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f72f49-e794-43d8-8a85-916ac622b95c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:43.303745 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f72f49-e794-43d8-8a85-916ac622b95c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:43.309731 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9559525-495e-4989-ab0a-de92e4896384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:43.310465 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9559525-495e-4989-ab0a-de92e4896384 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:43.310904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9559525-495e-4989-ab0a-de92e4896384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:43.311281 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9559525-495e-4989-ab0a-de92e4896384 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:43.312040 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9559525-495e-4989-ab0a-de92e4896384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:43.313784 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9559525-495e-4989-ab0a-de92e4896384 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:43.325568 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1036/2073] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:43.331278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de11e8-7eb9-4217-8580-39e8cd4596d6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:43.349899 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de11e8-7eb9-4217-8580-39e8cd4596d6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:43.355397 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:43.359178 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8853b3-cc93-41c8-b24a-8bf1768ab3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:43.359944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8853b3-cc93-41c8-b24a-8bf1768ab3bd admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:43.360249 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8853b3-cc93-41c8-b24a-8bf1768ab3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:43.360521 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8853b3-cc93-41c8-b24a-8bf1768ab3bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:43.361175 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8853b3-cc93-41c8-b24a-8bf1768ab3bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:43.363008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8853b3-cc93-41c8-b24a-8bf1768ab3bd admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:43.373625 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1038/2074] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:43.378215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9559525-495e-4989-ab0a-de92e4896384 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:43.388454 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9559525-495e-4989-ab0a-de92e4896384 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:43.393681 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0523d04-ef49-4806-8f85-19d1ab1dbb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:43.394335 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0523d04-ef49-4806-8f85-19d1ab1dbb51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:43.394613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0523d04-ef49-4806-8f85-19d1ab1dbb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:43.394920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0523d04-ef49-4806-8f85-19d1ab1dbb51 admin admin] PATH_INFO: `/v3/projects/c1dc0351a5154bfaab9c664adb97dac7/users/67951aee09b34949aa5971136e45b895/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:43.411334 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0523d04-ef49-4806-8f85-19d1ab1dbb51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1dc0351a5154bfaab9c664adb97dac7, user_id=67951aee09b34949aa5971136e45b895, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:43.414664 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0523d04-ef49-4806-8f85-19d1ab1dbb51 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:43.512793 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1037/2075] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:43 2022] PUT /identity/v3/projects/c1dc0351a5154bfaab9c664adb97dac7/users/67951aee09b34949aa5971136e45b895/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:33:43.523719 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267717e-cdbd-4750-b8f1-b1d9d7bcc081 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:43.524003 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267717e-cdbd-4750-b8f1-b1d9d7bcc081 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:43.524277 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267717e-cdbd-4750-b8f1-b1d9d7bcc081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:43.552595 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0267717e-cdbd-4750-b8f1-b1d9d7bcc081 None None] MFA Rules not processed for user `67951aee09b34949aa5971136e45b895`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:43.554095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0267717e-cdbd-4750-b8f1-b1d9d7bcc081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:43.625647 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1039/2076] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:43.636280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0523d04-ef49-4806-8f85-19d1ab1dbb51 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:43.698783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0523d04-ef49-4806-8f85-19d1ab1dbb51 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:43.703497 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a85ae-c2b6-403b-af73-8f0b198d28b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:43.704375 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a85ae-c2b6-403b-af73-8f0b198d28b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:43.704678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a85ae-c2b6-403b-af73-8f0b198d28b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:43.705178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5a85ae-c2b6-403b-af73-8f0b198d28b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:43.706248 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a85ae-c2b6-403b-af73-8f0b198d28b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:43.708275 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5a85ae-c2b6-403b-af73-8f0b198d28b5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:43.853972 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1038/2077] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:33:43 2022] POST /identity/v3/projects => generated 385 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:43.856117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267717e-cdbd-4750-b8f1-b1d9d7bcc081 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:43.868254 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267717e-cdbd-4750-b8f1-b1d9d7bcc081 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:43.927217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65e626-eb27-4847-b3d4-cbfb02290239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:43.927860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65e626-eb27-4847-b3d4-cbfb02290239 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:43.928117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65e626-eb27-4847-b3d4-cbfb02290239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:43.928884 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b65e626-eb27-4847-b3d4-cbfb02290239 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:43.929524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65e626-eb27-4847-b3d4-cbfb02290239 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:43.931480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b65e626-eb27-4847-b3d4-cbfb02290239 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:44.043960 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1040/2078] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:33:43 2022] POST /identity/v3/users => generated 299 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:33:44.048001 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a85ae-c2b6-403b-af73-8f0b198d28b5 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:44.058561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5a85ae-c2b6-403b-af73-8f0b198d28b5 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:44.063221 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e4a0b-f5ef-446d-8553-f61ffa78bd1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:44.063816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e4a0b-f5ef-446d-8553-f61ffa78bd1a admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:44.064093 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e4a0b-f5ef-446d-8553-f61ffa78bd1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:44.064363 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e4a0b-f5ef-446d-8553-f61ffa78bd1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:44.064997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e4a0b-f5ef-446d-8553-f61ffa78bd1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:44.066563 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e4a0b-f5ef-446d-8553-f61ffa78bd1a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:44.078016 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1039/2079] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:44.082281 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65e626-eb27-4847-b3d4-cbfb02290239 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:44.092066 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b65e626-eb27-4847-b3d4-cbfb02290239 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:44.097211 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016afae3-848d-4b8a-a465-1bea921a2627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:44.097878 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016afae3-848d-4b8a-a465-1bea921a2627 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:44.098165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016afae3-848d-4b8a-a465-1bea921a2627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:44.098444 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016afae3-848d-4b8a-a465-1bea921a2627 admin admin] PATH_INFO: `/v3/projects/ffaa9b3f24fe4791b4d5fcee47255451/users/7602c5419a85482191cae8015d380209/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:44.113553 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016afae3-848d-4b8a-a465-1bea921a2627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffaa9b3f24fe4791b4d5fcee47255451, user_id=7602c5419a85482191cae8015d380209, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:44.116692 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016afae3-848d-4b8a-a465-1bea921a2627 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:44.218786 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1041/2080] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:44 2022] PUT /identity/v3/projects/ffaa9b3f24fe4791b4d5fcee47255451/users/7602c5419a85482191cae8015d380209/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:44.222794 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e4a0b-f5ef-446d-8553-f61ffa78bd1a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:44.232890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e4a0b-f5ef-446d-8553-f61ffa78bd1a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:44.292019 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc97e62-7054-4620-9341-6a5dcc859b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:44.292019 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc97e62-7054-4620-9341-6a5dcc859b9a admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:44.292019 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc97e62-7054-4620-9341-6a5dcc859b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:44.292019 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc97e62-7054-4620-9341-6a5dcc859b9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:44.292367 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc97e62-7054-4620-9341-6a5dcc859b9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:44.293383 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc97e62-7054-4620-9341-6a5dcc859b9a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:44.307279 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1040/2081] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:44.308417 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016afae3-848d-4b8a-a465-1bea921a2627 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:44.319443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016afae3-848d-4b8a-a465-1bea921a2627 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:44.325493 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa751ec3-fde2-4d82-b869-acfd6d640ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:44.326325 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa751ec3-fde2-4d82-b869-acfd6d640ce1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:44.326856 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa751ec3-fde2-4d82-b869-acfd6d640ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:44.327566 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa751ec3-fde2-4d82-b869-acfd6d640ce1 admin admin] PATH_INFO: `/v3/projects/ffaa9b3f24fe4791b4d5fcee47255451/users/7602c5419a85482191cae8015d380209/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:44.329748 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa751ec3-fde2-4d82-b869-acfd6d640ce1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffaa9b3f24fe4791b4d5fcee47255451, user_id=7602c5419a85482191cae8015d380209, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:44.332576 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa751ec3-fde2-4d82-b869-acfd6d640ce1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:44.351307 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1042/2082] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:44 2022] PUT /identity/v3/projects/ffaa9b3f24fe4791b4d5fcee47255451/users/7602c5419a85482191cae8015d380209/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:44.362523 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9a079-7ecf-4341-b472-c433aad1aca5 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:44.362835 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9a079-7ecf-4341-b472-c433aad1aca5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:44.363128 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf9a079-7ecf-4341-b472-c433aad1aca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:44.392015 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1cf9a079-7ecf-4341-b472-c433aad1aca5 None None] MFA Rules not processed for user `7602c5419a85482191cae8015d380209`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:44.393270 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1cf9a079-7ecf-4341-b472-c433aad1aca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:44.465468 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:44.465468 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1041/2083] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:44 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:44.472890 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa751ec3-fde2-4d82-b869-acfd6d640ce1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:44.474629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-aa751ec3-fde2-4d82-b869-acfd6d640ce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:44.531010 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa751ec3-fde2-4d82-b869-acfd6d640ce1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:44.534684 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e5bb9-ad08-42f7-aba2-edab546f0ccd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:44.535319 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e5bb9-ad08-42f7-aba2-edab546f0ccd service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:44.535599 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e5bb9-ad08-42f7-aba2-edab546f0ccd service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:44.535868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e5bb9-ad08-42f7-aba2-edab546f0ccd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:44.547683 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:44.594576 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e5bb9-ad08-42f7-aba2-edab546f0ccd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:44.596030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e5bb9-ad08-42f7-aba2-edab546f0ccd service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:44.605658 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1043/2084] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:33:45.035952 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf9a079-7ecf-4341-b472-c433aad1aca5 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:45.036467 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1cf9a079-7ecf-4341-b472-c433aad1aca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:45.131664 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf9a079-7ecf-4341-b472-c433aad1aca5 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:45.137029 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224f9b9-d9d0-4525-a3d1-a2bd39a73ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:45.137708 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224f9b9-d9d0-4525-a3d1-a2bd39a73ab3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:45.137986 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224f9b9-d9d0-4525-a3d1-a2bd39a73ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:45.138254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224f9b9-d9d0-4525-a3d1-a2bd39a73ab3 admin admin] PATH_INFO: `/v3/users/fb347c98e8af4f3fac1f33b64aac6142` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:45.139382 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224f9b9-d9d0-4525-a3d1-a2bd39a73ab3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb347c98e8af4f3fac1f33b64aac6142)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:45.141388 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224f9b9-d9d0-4525-a3d1-a2bd39a73ab3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:45.395350 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8224f9b9-d9d0-4525-a3d1-a2bd39a73ab3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb347c98e8af4f3fac1f33b64aac6142'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:45.398794 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8224f9b9-d9d0-4525-a3d1-a2bd39a73ab3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb347c98e8af4f3fac1f33b64aac6142'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:45.489285 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8224f9b9-d9d0-4525-a3d1-a2bd39a73ab3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb347c98e8af4f3fac1f33b64aac6142'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:45.489918 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8224f9b9-d9d0-4525-a3d1-a2bd39a73ab3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb347c98e8af4f3fac1f33b64aac6142'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:45.504225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8224f9b9-d9d0-4525-a3d1-a2bd39a73ab3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb347c98e8af4f3fac1f33b64aac6142'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:45.511148 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1042/2085] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:33:45 2022] DELETE /identity/v3/users/fb347c98e8af4f3fac1f33b64aac6142 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:45.516252 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a7b99-63a2-4777-a9ac-58f287d3eec4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:45.518824 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a7b99-63a2-4777-a9ac-58f287d3eec4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:45.519783 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0a7b99-63a2-4777-a9ac-58f287d3eec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:45.524124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:45.551772 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5a0a7b99-63a2-4777-a9ac-58f287d3eec4 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:45.553292 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5a0a7b99-63a2-4777-a9ac-58f287d3eec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:45.627534 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1044/2086] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:45.639298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224f9b9-d9d0-4525-a3d1-a2bd39a73ab3 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:45.640914 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8224f9b9-d9d0-4525-a3d1-a2bd39a73ab3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:45.698739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224f9b9-d9d0-4525-a3d1-a2bd39a73ab3 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:45.702306 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a93aa-803d-41cf-adb0-a3f5317ec115 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:45.702932 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a93aa-803d-41cf-adb0-a3f5317ec115 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:45.703241 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a93aa-803d-41cf-adb0-a3f5317ec115 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:45.703532 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a93aa-803d-41cf-adb0-a3f5317ec115 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:45.712514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a93aa-803d-41cf-adb0-a3f5317ec115 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:45.713926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a93aa-803d-41cf-adb0-a3f5317ec115 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:45.723291 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1043/2087] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:33:45.781308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:46.161523 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a7b99-63a2-4777-a9ac-58f287d3eec4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:46.163079 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5a0a7b99-63a2-4777-a9ac-58f287d3eec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:46.179023 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0a7b99-63a2-4777-a9ac-58f287d3eec4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:46.184081 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63525e9-edf0-45ff-8f7e-5a1047fd37d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:46.185144 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63525e9-edf0-45ff-8f7e-5a1047fd37d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:46.185144 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63525e9-edf0-45ff-8f7e-5a1047fd37d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:46.185344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63525e9-edf0-45ff-8f7e-5a1047fd37d2 admin admin] PATH_INFO: `/v3/projects/1a822457cfef430eb9869c1b0ed23e00` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:46.186627 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63525e9-edf0-45ff-8f7e-5a1047fd37d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a822457cfef430eb9869c1b0ed23e00)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:46.188559 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63525e9-edf0-45ff-8f7e-5a1047fd37d2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:46.301411 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e63525e9-edf0-45ff-8f7e-5a1047fd37d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a822457cfef430eb9869c1b0ed23e00', 'request_id': 'req-e63525e9-edf0-45ff-8f7e-5a1047fd37d2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:46.306845 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e63525e9-edf0-45ff-8f7e-5a1047fd37d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a822457cfef430eb9869c1b0ed23e00', 'request_id': 'req-e63525e9-edf0-45ff-8f7e-5a1047fd37d2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:46.314695 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e63525e9-edf0-45ff-8f7e-5a1047fd37d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a822457cfef430eb9869c1b0ed23e00', 'request_id': 'req-e63525e9-edf0-45ff-8f7e-5a1047fd37d2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:46.355882 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:46.384479 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e63525e9-edf0-45ff-8f7e-5a1047fd37d2 admin admin] The token cache is being invalidate because project 1a822457cfef430eb9869c1b0ed23e00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:33:46.384773 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e63525e9-edf0-45ff-8f7e-5a1047fd37d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:46.385414 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e63525e9-edf0-45ff-8f7e-5a1047fd37d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:46.387015 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1045/2088] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:33:46 2022] DELETE /identity/v3/projects/1a822457cfef430eb9869c1b0ed23e00 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:46.390894 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a93aa-803d-41cf-adb0-a3f5317ec115 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:46.400987 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a93aa-803d-41cf-adb0-a3f5317ec115 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:46.462034 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d496a54-d0eb-4018-a993-aedf7c10c318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:46.462827 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d496a54-d0eb-4018-a993-aedf7c10c318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:46.462984 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d496a54-d0eb-4018-a993-aedf7c10c318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:46.463261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d496a54-d0eb-4018-a993-aedf7c10c318 admin admin] PATH_INFO: `/v3/users/591ca22cdf194d969dd85604ffb84c6b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:46.464379 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d496a54-d0eb-4018-a993-aedf7c10c318 admin admin] RBAC: Authorizing `identity:delete_user(user_id=591ca22cdf194d969dd85604ffb84c6b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:46.466291 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d496a54-d0eb-4018-a993-aedf7c10c318 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:46.645411 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5d496a54-d0eb-4018-a993-aedf7c10c318 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '591ca22cdf194d969dd85604ffb84c6b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:46.652830 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5d496a54-d0eb-4018-a993-aedf7c10c318 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '591ca22cdf194d969dd85604ffb84c6b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:46.711934 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5d496a54-d0eb-4018-a993-aedf7c10c318 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '591ca22cdf194d969dd85604ffb84c6b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:46.712555 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5d496a54-d0eb-4018-a993-aedf7c10c318 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '591ca22cdf194d969dd85604ffb84c6b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:46.724712 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5d496a54-d0eb-4018-a993-aedf7c10c318 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '591ca22cdf194d969dd85604ffb84c6b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:46.730456 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1044/2089] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:33:46 2022] DELETE /identity/v3/users/591ca22cdf194d969dd85604ffb84c6b => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:46.741215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:47.066513 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:47.185657 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63525e9-edf0-45ff-8f7e-5a1047fd37d2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:47.185657 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e63525e9-edf0-45ff-8f7e-5a1047fd37d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:47.197328 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63525e9-edf0-45ff-8f7e-5a1047fd37d2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:47.282269 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d1bf74-ff8a-4fec-8a3b-947ad30400c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:47.286262 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1bf74-ff8a-4fec-8a3b-947ad30400c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:47.286262 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1bf74-ff8a-4fec-8a3b-947ad30400c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:47.286262 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d1bf74-ff8a-4fec-8a3b-947ad30400c6 admin admin] PATH_INFO: `/v3/projects/d8605b7ac11c402680ec5bd1709cadec` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:47.286262 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1bf74-ff8a-4fec-8a3b-947ad30400c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8605b7ac11c402680ec5bd1709cadec)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:47.286657 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d1bf74-ff8a-4fec-8a3b-947ad30400c6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:47.391708 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b8d1bf74-ff8a-4fec-8a3b-947ad30400c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8605b7ac11c402680ec5bd1709cadec', 'request_id': 'req-b8d1bf74-ff8a-4fec-8a3b-947ad30400c6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:47.396941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b8d1bf74-ff8a-4fec-8a3b-947ad30400c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8605b7ac11c402680ec5bd1709cadec', 'request_id': 'req-b8d1bf74-ff8a-4fec-8a3b-947ad30400c6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:47.402985 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b8d1bf74-ff8a-4fec-8a3b-947ad30400c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8605b7ac11c402680ec5bd1709cadec', 'request_id': 'req-b8d1bf74-ff8a-4fec-8a3b-947ad30400c6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:47.486515 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b8d1bf74-ff8a-4fec-8a3b-947ad30400c6 admin admin] The token cache is being invalidate because project d8605b7ac11c402680ec5bd1709cadec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:33:47.486848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b8d1bf74-ff8a-4fec-8a3b-947ad30400c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:47.487467 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b8d1bf74-ff8a-4fec-8a3b-947ad30400c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:47.488933 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1046/2090] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:33:47 2022] DELETE /identity/v3/projects/d8605b7ac11c402680ec5bd1709cadec => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:47.495590 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb190b5-d6af-4539-bc38-7434383d71b6 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:47.496045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb190b5-d6af-4539-bc38-7434383d71b6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:47.496151 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb190b5-d6af-4539-bc38-7434383d71b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:47.515736 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6bb190b5-d6af-4539-bc38-7434383d71b6 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:47.516943 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6bb190b5-d6af-4539-bc38-7434383d71b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:47.576493 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1045/2091] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:47.584302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10a894-1f9f-479e-953b-9b27c09cfe68 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:47.584557 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10a894-1f9f-479e-953b-9b27c09cfe68 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:47.584857 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10a894-1f9f-479e-953b-9b27c09cfe68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:47.584977 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:47.605002 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4d10a894-1f9f-479e-953b-9b27c09cfe68 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:47.606220 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4d10a894-1f9f-479e-953b-9b27c09cfe68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:47.614045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1047/2092] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:33:47.620844 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb190b5-d6af-4539-bc38-7434383d71b6 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:47.630535 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb190b5-d6af-4539-bc38-7434383d71b6 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:47.635341 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898003af-3da9-41e5-9f38-81e1f21628f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:47.635934 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898003af-3da9-41e5-9f38-81e1f21628f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:47.636212 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898003af-3da9-41e5-9f38-81e1f21628f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:47.636483 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898003af-3da9-41e5-9f38-81e1f21628f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:47.637131 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898003af-3da9-41e5-9f38-81e1f21628f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:33:47.637427 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898003af-3da9-41e5-9f38-81e1f21628f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:47.639043 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898003af-3da9-41e5-9f38-81e1f21628f5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:47.651068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1046/2093] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:33:47.655289 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10a894-1f9f-479e-953b-9b27c09cfe68 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:47.664956 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10a894-1f9f-479e-953b-9b27c09cfe68 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:47.670015 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e881791c-0421-496a-a821-1b7b4a60531b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:47.670197 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881791c-0421-496a-a821-1b7b4a60531b admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:47.670435 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881791c-0421-496a-a821-1b7b4a60531b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:47.670731 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e881791c-0421-496a-a821-1b7b4a60531b admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:47.671409 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e881791c-0421-496a-a821-1b7b4a60531b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:47.673270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e881791c-0421-496a-a821-1b7b4a60531b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:47.764453 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1048/2094] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:33:47 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:47.768913 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898003af-3da9-41e5-9f38-81e1f21628f5 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:47.779939 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898003af-3da9-41e5-9f38-81e1f21628f5 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:47.834377 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa737210-255c-4a1b-859b-177fe64d85e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:47.835535 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa737210-255c-4a1b-859b-177fe64d85e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:47.835817 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa737210-255c-4a1b-859b-177fe64d85e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:47.836088 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa737210-255c-4a1b-859b-177fe64d85e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:47.836720 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa737210-255c-4a1b-859b-177fe64d85e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:47.838624 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa737210-255c-4a1b-859b-177fe64d85e3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:47.855106 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:47.918623 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1047/2095] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:33:47 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:33:47.922967 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e881791c-0421-496a-a821-1b7b4a60531b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:47.933435 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e881791c-0421-496a-a821-1b7b4a60531b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:47.938124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651fbdfa-d2c1-408c-9455-b6052df4ec28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:47.938747 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fbdfa-d2c1-408c-9455-b6052df4ec28 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:47.939032 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fbdfa-d2c1-408c-9455-b6052df4ec28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:47.939305 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651fbdfa-d2c1-408c-9455-b6052df4ec28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:47.940002 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651fbdfa-d2c1-408c-9455-b6052df4ec28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:47.941592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651fbdfa-d2c1-408c-9455-b6052df4ec28 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:47.951675 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1049/2096] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:47.955478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa737210-255c-4a1b-859b-177fe64d85e3 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:47.966262 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa737210-255c-4a1b-859b-177fe64d85e3 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:47.971041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c61230-7187-4fb1-97d9-3e82b4ec1d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:47.971627 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c61230-7187-4fb1-97d9-3e82b4ec1d59 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:47.971903 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c61230-7187-4fb1-97d9-3e82b4ec1d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:47.972173 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c61230-7187-4fb1-97d9-3e82b4ec1d59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:47.972804 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c61230-7187-4fb1-97d9-3e82b4ec1d59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:47.974462 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c61230-7187-4fb1-97d9-3e82b4ec1d59 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:47.985920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1048/2097] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:47.987394 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:47.989606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651fbdfa-d2c1-408c-9455-b6052df4ec28 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:47.999674 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651fbdfa-d2c1-408c-9455-b6052df4ec28 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:48.004606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8676b6-6b74-4565-9477-c6b6dc356a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:48.005251 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8676b6-6b74-4565-9477-c6b6dc356a21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:48.005534 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8676b6-6b74-4565-9477-c6b6dc356a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:48.005810 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8676b6-6b74-4565-9477-c6b6dc356a21 admin admin] PATH_INFO: `/v3/projects/5f36d1f66af34138a37b4dff38a8e2a9/users/222f250fd245418f85ddb4c5eb368636/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:48.020423 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8676b6-6b74-4565-9477-c6b6dc356a21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f36d1f66af34138a37b4dff38a8e2a9, user_id=222f250fd245418f85ddb4c5eb368636, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:48.026429 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8676b6-6b74-4565-9477-c6b6dc356a21 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:48.094131 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1050/2098] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:47 2022] PUT /identity/v3/projects/5f36d1f66af34138a37b4dff38a8e2a9/users/222f250fd245418f85ddb4c5eb368636/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:48.105351 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aecce7-f10d-444d-91eb-6b8df82c642d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:48.105632 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aecce7-f10d-444d-91eb-6b8df82c642d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:48.105904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aecce7-f10d-444d-91eb-6b8df82c642d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:48.133758 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-23aecce7-f10d-444d-91eb-6b8df82c642d None None] MFA Rules not processed for user `222f250fd245418f85ddb4c5eb368636`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:48.134983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-23aecce7-f10d-444d-91eb-6b8df82c642d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:48.169313 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:48.196234 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1049/2099] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:48.200792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8676b6-6b74-4565-9477-c6b6dc356a21 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:48.203451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:48.267679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8676b6-6b74-4565-9477-c6b6dc356a21 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:48.276033 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23030ac1-b632-4c9e-b9c0-3ab79c5e4401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:48.276659 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23030ac1-b632-4c9e-b9c0-3ab79c5e4401 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:48.276940 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23030ac1-b632-4c9e-b9c0-3ab79c5e4401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:48.277213 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23030ac1-b632-4c9e-b9c0-3ab79c5e4401 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:48.277866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23030ac1-b632-4c9e-b9c0-3ab79c5e4401 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:48.280070 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23030ac1-b632-4c9e-b9c0-3ab79c5e4401 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:48.368953 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1051/2100] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:33:48 2022] POST /identity/v3/projects => generated 381 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:48.373205 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aecce7-f10d-444d-91eb-6b8df82c642d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:48.385187 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aecce7-f10d-444d-91eb-6b8df82c642d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:48.418858 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:48.441316 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d457e7-650a-4c29-aa3d-4a9e3bec3707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:48.442000 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d457e7-650a-4c29-aa3d-4a9e3bec3707 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:48.442861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d457e7-650a-4c29-aa3d-4a9e3bec3707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:48.445496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d457e7-650a-4c29-aa3d-4a9e3bec3707 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:48.445496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d457e7-650a-4c29-aa3d-4a9e3bec3707 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:48.446373 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d457e7-650a-4c29-aa3d-4a9e3bec3707 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:48.530601 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1050/2101] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:33:48 2022] POST /identity/v3/users => generated 297 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:48.530601 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:48.532523 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23030ac1-b632-4c9e-b9c0-3ab79c5e4401 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:48.542922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23030ac1-b632-4c9e-b9c0-3ab79c5e4401 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:48.549009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d87059-0e41-43da-a212-39650bf8781d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:48.549009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d87059-0e41-43da-a212-39650bf8781d admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:48.549009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d87059-0e41-43da-a212-39650bf8781d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:48.549009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d87059-0e41-43da-a212-39650bf8781d admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:48.549377 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d87059-0e41-43da-a212-39650bf8781d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:48.550731 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d87059-0e41-43da-a212-39650bf8781d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:48.561674 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1052/2102] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:48.568045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d457e7-650a-4c29-aa3d-4a9e3bec3707 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:48.579671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d457e7-650a-4c29-aa3d-4a9e3bec3707 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:48.584362 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688295d5-c8e5-45bc-bf34-9dd93903770c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:48.585345 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688295d5-c8e5-45bc-bf34-9dd93903770c admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:48.585629 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688295d5-c8e5-45bc-bf34-9dd93903770c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:48.585907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688295d5-c8e5-45bc-bf34-9dd93903770c admin admin] PATH_INFO: `/v3/projects/81e87a6b57784e80a73a2f0047e02eb5/users/64350c289fb7474190a773ccea307463/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:48.586781 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:48.601514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688295d5-c8e5-45bc-bf34-9dd93903770c admin admin] RBAC: Authorizing `identity:create_grant(project_id=81e87a6b57784e80a73a2f0047e02eb5, user_id=64350c289fb7474190a773ccea307463, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:48.604258 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688295d5-c8e5-45bc-bf34-9dd93903770c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:48.676177 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1051/2103] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:48 2022] PUT /identity/v3/projects/81e87a6b57784e80a73a2f0047e02eb5/users/64350c289fb7474190a773ccea307463/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:33:48.680601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d87059-0e41-43da-a212-39650bf8781d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:48.690658 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d87059-0e41-43da-a212-39650bf8781d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:48.745420 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964ff55-86e7-434e-b239-d6636e3b79f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:48.746488 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964ff55-86e7-434e-b239-d6636e3b79f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:48.747439 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964ff55-86e7-434e-b239-d6636e3b79f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:48.747753 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964ff55-86e7-434e-b239-d6636e3b79f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:48.748417 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1964ff55-86e7-434e-b239-d6636e3b79f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:48.750035 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1964ff55-86e7-434e-b239-d6636e3b79f1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:48.760818 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1053/2104] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:48.765220 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688295d5-c8e5-45bc-bf34-9dd93903770c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:48.773432 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:48.777389 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688295d5-c8e5-45bc-bf34-9dd93903770c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:48.781893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfe7a88-ad06-4b06-a23c-9d50b9456de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:48.782575 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe7a88-ad06-4b06-a23c-9d50b9456de0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:48.782886 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe7a88-ad06-4b06-a23c-9d50b9456de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:48.783162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe7a88-ad06-4b06-a23c-9d50b9456de0 admin admin] PATH_INFO: `/v3/projects/81e87a6b57784e80a73a2f0047e02eb5/users/64350c289fb7474190a773ccea307463/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:48.785057 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfe7a88-ad06-4b06-a23c-9d50b9456de0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81e87a6b57784e80a73a2f0047e02eb5, user_id=64350c289fb7474190a773ccea307463, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:48.787739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfe7a88-ad06-4b06-a23c-9d50b9456de0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:48.796091 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:48.798315 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1052/2105] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:48 2022] PUT /identity/v3/projects/81e87a6b57784e80a73a2f0047e02eb5/users/64350c289fb7474190a773ccea307463/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:48.809128 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50621e-37c5-498a-9dab-65b5a7541ad1 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:48.809417 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50621e-37c5-498a-9dab-65b5a7541ad1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:48.809696 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50621e-37c5-498a-9dab-65b5a7541ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:48.836459 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9f50621e-37c5-498a-9dab-65b5a7541ad1 None None] MFA Rules not processed for user `64350c289fb7474190a773ccea307463`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:48.837662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9f50621e-37c5-498a-9dab-65b5a7541ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:48.921836 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1054/2106] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:48 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:48.923756 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfe7a88-ad06-4b06-a23c-9d50b9456de0 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:48.925444 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fbfe7a88-ad06-4b06-a23c-9d50b9456de0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:48.982129 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfe7a88-ad06-4b06-a23c-9d50b9456de0 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:48.985734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4feecc-2f40-4fe3-bca1-5506ee49f7b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:48.986333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4feecc-2f40-4fe3-bca1-5506ee49f7b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:48.986613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4feecc-2f40-4fe3-bca1-5506ee49f7b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:48.986932 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4feecc-2f40-4fe3-bca1-5506ee49f7b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:48.996351 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4feecc-2f40-4fe3-bca1-5506ee49f7b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:48.997708 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4feecc-2f40-4fe3-bca1-5506ee49f7b8 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:49.007186 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1053/2107] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:48 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:33:49.586898 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50621e-37c5-498a-9dab-65b5a7541ad1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:49.650243 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50621e-37c5-498a-9dab-65b5a7541ad1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:49.654726 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680308cb-5e36-41b5-8a41-49c6dfb47d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:49.655353 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680308cb-5e36-41b5-8a41-49c6dfb47d2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:49.655633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680308cb-5e36-41b5-8a41-49c6dfb47d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:49.655906 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680308cb-5e36-41b5-8a41-49c6dfb47d2a admin admin] PATH_INFO: `/v3/users/222f250fd245418f85ddb4c5eb368636` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:49.657006 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680308cb-5e36-41b5-8a41-49c6dfb47d2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=222f250fd245418f85ddb4c5eb368636)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:49.658942 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680308cb-5e36-41b5-8a41-49c6dfb47d2a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:49.687221 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:49.869158 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-680308cb-5e36-41b5-8a41-49c6dfb47d2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '222f250fd245418f85ddb4c5eb368636'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:49.951987 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-680308cb-5e36-41b5-8a41-49c6dfb47d2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '222f250fd245418f85ddb4c5eb368636'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:49.952604 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-680308cb-5e36-41b5-8a41-49c6dfb47d2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '222f250fd245418f85ddb4c5eb368636'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:49.953231 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-680308cb-5e36-41b5-8a41-49c6dfb47d2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '222f250fd245418f85ddb4c5eb368636'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:49.966156 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-680308cb-5e36-41b5-8a41-49c6dfb47d2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '222f250fd245418f85ddb4c5eb368636'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:49.979011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1055/2108] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:33:49 2022] DELETE /identity/v3/users/222f250fd245418f85ddb4c5eb368636 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:49.983467 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787d00b-442e-4796-9e8e-a66a2e4eeeb9 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:49.983931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787d00b-442e-4796-9e8e-a66a2e4eeeb9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:49.984048 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0787d00b-442e-4796-9e8e-a66a2e4eeeb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:50.003124 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0787d00b-442e-4796-9e8e-a66a2e4eeeb9 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:50.004329 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0787d00b-442e-4796-9e8e-a66a2e4eeeb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:50.065889 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1054/2109] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:50.075508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680308cb-5e36-41b5-8a41-49c6dfb47d2a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:50.076900 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-680308cb-5e36-41b5-8a41-49c6dfb47d2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:50.132600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680308cb-5e36-41b5-8a41-49c6dfb47d2a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:50.136048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d8b6f-9cca-47d9-a238-1caca5605183 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:50.139855 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d8b6f-9cca-47d9-a238-1caca5605183 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:50.139855 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d8b6f-9cca-47d9-a238-1caca5605183 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:50.139855 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d8b6f-9cca-47d9-a238-1caca5605183 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:50.145856 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259d8b6f-9cca-47d9-a238-1caca5605183 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:50.147819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259d8b6f-9cca-47d9-a238-1caca5605183 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:50.156954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1056/2110] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:33:50.580119 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0787d00b-442e-4796-9e8e-a66a2e4eeeb9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:50.580119 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0787d00b-442e-4796-9e8e-a66a2e4eeeb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:50.592033 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0787d00b-442e-4796-9e8e-a66a2e4eeeb9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:50.607543 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f3bfb-f2dc-44d9-9207-acf539f46121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:50.608968 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f3bfb-f2dc-44d9-9207-acf539f46121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:50.609698 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f3bfb-f2dc-44d9-9207-acf539f46121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:50.610214 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f3bfb-f2dc-44d9-9207-acf539f46121 admin admin] PATH_INFO: `/v3/projects/5f36d1f66af34138a37b4dff38a8e2a9` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:50.611959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f3bfb-f2dc-44d9-9207-acf539f46121 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f36d1f66af34138a37b4dff38a8e2a9)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:50.614207 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f3bfb-f2dc-44d9-9207-acf539f46121 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:50.616767 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:50.742561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-956f3bfb-f2dc-44d9-9207-acf539f46121 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f36d1f66af34138a37b4dff38a8e2a9', 'request_id': 'req-956f3bfb-f2dc-44d9-9207-acf539f46121'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:50.801092 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-956f3bfb-f2dc-44d9-9207-acf539f46121 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f36d1f66af34138a37b4dff38a8e2a9', 'request_id': 'req-956f3bfb-f2dc-44d9-9207-acf539f46121'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:50.810543 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-956f3bfb-f2dc-44d9-9207-acf539f46121 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f36d1f66af34138a37b4dff38a8e2a9', 'request_id': 'req-956f3bfb-f2dc-44d9-9207-acf539f46121'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:50.822309 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-956f3bfb-f2dc-44d9-9207-acf539f46121 admin admin] The token cache is being invalidate because project 5f36d1f66af34138a37b4dff38a8e2a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:33:50.822603 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-956f3bfb-f2dc-44d9-9207-acf539f46121 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:50.823296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-956f3bfb-f2dc-44d9-9207-acf539f46121 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:50.825145 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1055/2111] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:33:50 2022] DELETE /identity/v3/projects/5f36d1f66af34138a37b4dff38a8e2a9 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:50.829532 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d8b6f-9cca-47d9-a238-1caca5605183 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:50.840153 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259d8b6f-9cca-47d9-a238-1caca5605183 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:50.898200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c4ad62-ed90-44d1-8366-d7f249b87c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:50.898772 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4ad62-ed90-44d1-8366-d7f249b87c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:50.899052 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4ad62-ed90-44d1-8366-d7f249b87c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:50.899320 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4ad62-ed90-44d1-8366-d7f249b87c66 admin admin] PATH_INFO: `/v3/users/64350c289fb7474190a773ccea307463` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:50.900424 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c4ad62-ed90-44d1-8366-d7f249b87c66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64350c289fb7474190a773ccea307463)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:50.902312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c4ad62-ed90-44d1-8366-d7f249b87c66 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:51.035843 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f3bfb-f2dc-44d9-9207-acf539f46121 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:51.037265 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-956f3bfb-f2dc-44d9-9207-acf539f46121 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:51.094032 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f3bfb-f2dc-44d9-9207-acf539f46121 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:51.097589 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df26ad-4ffb-45a0-b29d-41a0bcb0d02d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:51.098190 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df26ad-4ffb-45a0-b29d-41a0bcb0d02d service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:51.098478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df26ad-4ffb-45a0-b29d-41a0bcb0d02d service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:51.098772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df26ad-4ffb-45a0-b29d-41a0bcb0d02d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:51.101052 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-94df26ad-4ffb-45a0-b29d-41a0bcb0d02d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:51.143530 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:51.145268 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-32c4ad62-ed90-44d1-8366-d7f249b87c66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64350c289fb7474190a773ccea307463'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:51.171413 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df26ad-4ffb-45a0-b29d-41a0bcb0d02d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:51.173202 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df26ad-4ffb-45a0-b29d-41a0bcb0d02d service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:51.185139 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1056/2112] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:33:51.260267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-32c4ad62-ed90-44d1-8366-d7f249b87c66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64350c289fb7474190a773ccea307463'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:51.265243 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-32c4ad62-ed90-44d1-8366-d7f249b87c66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64350c289fb7474190a773ccea307463'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:51.265243 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-32c4ad62-ed90-44d1-8366-d7f249b87c66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64350c289fb7474190a773ccea307463'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:51.274486 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-32c4ad62-ed90-44d1-8366-d7f249b87c66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64350c289fb7474190a773ccea307463'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:51.286334 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1057/2113] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:33:50 2022] DELETE /identity/v3/users/64350c289fb7474190a773ccea307463 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:33:51.742336 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df26ad-4ffb-45a0-b29d-41a0bcb0d02d service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:51.742336 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-94df26ad-4ffb-45a0-b29d-41a0bcb0d02d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:51.756353 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df26ad-4ffb-45a0-b29d-41a0bcb0d02d service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:51.830292 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:51.844038 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132ad86-7ab3-453c-9e44-ca07a2d77fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:51.846988 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132ad86-7ab3-453c-9e44-ca07a2d77fce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:51.846988 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132ad86-7ab3-453c-9e44-ca07a2d77fce admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:51.846988 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132ad86-7ab3-453c-9e44-ca07a2d77fce admin admin] PATH_INFO: `/v3/projects/81e87a6b57784e80a73a2f0047e02eb5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:51.846988 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c132ad86-7ab3-453c-9e44-ca07a2d77fce admin admin] RBAC: Authorizing `identity:delete_project(project_id=81e87a6b57784e80a73a2f0047e02eb5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:51.848882 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c132ad86-7ab3-453c-9e44-ca07a2d77fce admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:51.960837 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c132ad86-7ab3-453c-9e44-ca07a2d77fce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81e87a6b57784e80a73a2f0047e02eb5', 'request_id': 'req-c132ad86-7ab3-453c-9e44-ca07a2d77fce'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:52.031569 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c132ad86-7ab3-453c-9e44-ca07a2d77fce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81e87a6b57784e80a73a2f0047e02eb5', 'request_id': 'req-c132ad86-7ab3-453c-9e44-ca07a2d77fce'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:52.037546 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c132ad86-7ab3-453c-9e44-ca07a2d77fce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81e87a6b57784e80a73a2f0047e02eb5', 'request_id': 'req-c132ad86-7ab3-453c-9e44-ca07a2d77fce'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:52.048991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c132ad86-7ab3-453c-9e44-ca07a2d77fce admin admin] The token cache is being invalidate because project 81e87a6b57784e80a73a2f0047e02eb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:33:52.049277 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c132ad86-7ab3-453c-9e44-ca07a2d77fce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:52.049925 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c132ad86-7ab3-453c-9e44-ca07a2d77fce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:52.051449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1057/2114] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:33:51 2022] DELETE /identity/v3/projects/81e87a6b57784e80a73a2f0047e02eb5 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:52.056916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddff92-a53e-4cfb-805d-868972badb4e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:52.057341 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddff92-a53e-4cfb-805d-868972badb4e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:52.057470 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddff92-a53e-4cfb-805d-868972badb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:52.076736 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-44ddff92-a53e-4cfb-805d-868972badb4e None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:52.077987 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-44ddff92-a53e-4cfb-805d-868972badb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:52.137134 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1058/2115] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:52.145811 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160bbc9-ab74-491d-9bb1-2cdeb18ba259 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:52.146094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160bbc9-ab74-491d-9bb1-2cdeb18ba259 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:52.146367 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160bbc9-ab74-491d-9bb1-2cdeb18ba259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:52.166847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-9160bbc9-ab74-491d-9bb1-2cdeb18ba259 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:52.168045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9160bbc9-ab74-491d-9bb1-2cdeb18ba259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:52.175849 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1058/2116] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:52.183309 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddff92-a53e-4cfb-805d-868972badb4e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:52.193042 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddff92-a53e-4cfb-805d-868972badb4e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:52.197599 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee847ea5-38ea-4621-80ab-7fcebb71f670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:52.198203 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee847ea5-38ea-4621-80ab-7fcebb71f670 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:52.198483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee847ea5-38ea-4621-80ab-7fcebb71f670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:52.198778 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee847ea5-38ea-4621-80ab-7fcebb71f670 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:52.199430 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee847ea5-38ea-4621-80ab-7fcebb71f670 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:33:52.199736 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee847ea5-38ea-4621-80ab-7fcebb71f670 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:52.201317 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee847ea5-38ea-4621-80ab-7fcebb71f670 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:52.213732 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1059/2117] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:33:52.217995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160bbc9-ab74-491d-9bb1-2cdeb18ba259 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:52.227935 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160bbc9-ab74-491d-9bb1-2cdeb18ba259 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:52.232912 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de67720-29ea-452a-b98c-6960f08bb985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:52.233515 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de67720-29ea-452a-b98c-6960f08bb985 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:52.233797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de67720-29ea-452a-b98c-6960f08bb985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:52.234074 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de67720-29ea-452a-b98c-6960f08bb985 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:52.234722 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de67720-29ea-452a-b98c-6960f08bb985 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:52.236671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de67720-29ea-452a-b98c-6960f08bb985 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:52.390858 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1059/2118] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:33:52 2022] POST /identity/v3/projects => generated 381 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:52.394989 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee847ea5-38ea-4621-80ab-7fcebb71f670 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:52.405491 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee847ea5-38ea-4621-80ab-7fcebb71f670 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:52.461344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a491d-d7cb-4451-9295-98aecb93f937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:52.461939 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a491d-d7cb-4451-9295-98aecb93f937 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:52.462265 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a491d-d7cb-4451-9295-98aecb93f937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:52.462534 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a491d-d7cb-4451-9295-98aecb93f937 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:52.463220 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5a491d-d7cb-4451-9295-98aecb93f937 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:52.465132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5a491d-d7cb-4451-9295-98aecb93f937 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:52.546500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1060/2119] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:33:52 2022] POST /identity/v3/users => generated 291 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:52.553488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de67720-29ea-452a-b98c-6960f08bb985 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:52.563622 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de67720-29ea-452a-b98c-6960f08bb985 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:52.568434 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7825f28-9698-45b2-ac60-c80defe79d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:52.569922 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7825f28-9698-45b2-ac60-c80defe79d33 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:52.570201 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7825f28-9698-45b2-ac60-c80defe79d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:52.570468 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7825f28-9698-45b2-ac60-c80defe79d33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:52.571136 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7825f28-9698-45b2-ac60-c80defe79d33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:52.572751 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7825f28-9698-45b2-ac60-c80defe79d33 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:52.587661 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1060/2120] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:52.590540 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a491d-d7cb-4451-9295-98aecb93f937 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:52.600606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a491d-d7cb-4451-9295-98aecb93f937 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:52.605223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767e96f-612e-4803-aa73-1267f286075f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:52.605835 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767e96f-612e-4803-aa73-1267f286075f admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:52.606115 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767e96f-612e-4803-aa73-1267f286075f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:52.606382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767e96f-612e-4803-aa73-1267f286075f admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:52.607094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767e96f-612e-4803-aa73-1267f286075f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:52.608713 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767e96f-612e-4803-aa73-1267f286075f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:52.619200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1061/2121] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:52.623164 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7825f28-9698-45b2-ac60-c80defe79d33 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:52.633760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7825f28-9698-45b2-ac60-c80defe79d33 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:52.638334 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbf4b5-d35c-4b49-8efc-dce4b00159d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:52.639006 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbf4b5-d35c-4b49-8efc-dce4b00159d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:52.639284 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbf4b5-d35c-4b49-8efc-dce4b00159d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:52.639581 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbf4b5-d35c-4b49-8efc-dce4b00159d8 admin admin] PATH_INFO: `/v3/projects/cfd8100a870944c9a2c44523a8791469/users/42b3f8a8dc7b4a6d9c68d41f07035b68/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:52.654259 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bbf4b5-d35c-4b49-8efc-dce4b00159d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfd8100a870944c9a2c44523a8791469, user_id=42b3f8a8dc7b4a6d9c68d41f07035b68, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:52.656965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bbf4b5-d35c-4b49-8efc-dce4b00159d8 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:52.741233 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1061/2122] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:52 2022] PUT /identity/v3/projects/cfd8100a870944c9a2c44523a8791469/users/42b3f8a8dc7b4a6d9c68d41f07035b68/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:52.751520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd42382-247d-432b-a216-2df231265330 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:52.751884 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd42382-247d-432b-a216-2df231265330 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:52.753564 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd42382-247d-432b-a216-2df231265330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:52.780604 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4bd42382-247d-432b-a216-2df231265330 None None] MFA Rules not processed for user `42b3f8a8dc7b4a6d9c68d41f07035b68`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:52.781804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4bd42382-247d-432b-a216-2df231265330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:52.797849 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:52.844342 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1062/2123] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:52.848929 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbf4b5-d35c-4b49-8efc-dce4b00159d8 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:52.917482 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbf4b5-d35c-4b49-8efc-dce4b00159d8 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:52.924443 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd91f8-aacb-492a-8bd9-a67fc017b291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:52.925104 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd91f8-aacb-492a-8bd9-a67fc017b291 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:52.925391 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd91f8-aacb-492a-8bd9-a67fc017b291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:52.925698 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd91f8-aacb-492a-8bd9-a67fc017b291 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:52.926598 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd91f8-aacb-492a-8bd9-a67fc017b291 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:52.929804 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acd91f8-aacb-492a-8bd9-a67fc017b291 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:53.055188 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:53.063944 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1062/2124] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:33:52 2022] POST /identity/v3/projects => generated 381 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:53.069232 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd42382-247d-432b-a216-2df231265330 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:53.079388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd42382-247d-432b-a216-2df231265330 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:53.137910 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8afef-6dc8-4e00-aa7e-c017a93b6037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:53.138524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8afef-6dc8-4e00-aa7e-c017a93b6037 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:53.138846 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8afef-6dc8-4e00-aa7e-c017a93b6037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:53.139116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a8afef-6dc8-4e00-aa7e-c017a93b6037 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:53.139740 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8afef-6dc8-4e00-aa7e-c017a93b6037 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:53.141628 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a8afef-6dc8-4e00-aa7e-c017a93b6037 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:53.258951 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1063/2125] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:33:53 2022] POST /identity/v3/users => generated 297 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:33:53.262831 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd91f8-aacb-492a-8bd9-a67fc017b291 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:53.273543 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd91f8-aacb-492a-8bd9-a67fc017b291 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:53.278091 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052476f-0462-4dff-a7ea-c478235934e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:53.278687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052476f-0462-4dff-a7ea-c478235934e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:53.279000 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052476f-0462-4dff-a7ea-c478235934e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:53.279274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d052476f-0462-4dff-a7ea-c478235934e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:53.279923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d052476f-0462-4dff-a7ea-c478235934e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:53.281623 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d052476f-0462-4dff-a7ea-c478235934e3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:53.293749 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1063/2126] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:53.298016 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8afef-6dc8-4e00-aa7e-c017a93b6037 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:53.308177 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a8afef-6dc8-4e00-aa7e-c017a93b6037 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:53.312908 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0827aac-46b5-4753-9b41-4958d7ae603a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:53.313558 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0827aac-46b5-4753-9b41-4958d7ae603a admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:53.313838 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0827aac-46b5-4753-9b41-4958d7ae603a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:53.314112 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0827aac-46b5-4753-9b41-4958d7ae603a admin admin] PATH_INFO: `/v3/projects/e527489d37524b42ae9dbc3f8d4f8e7d/users/1c4515a7d557456a9fa71105ee5142ef/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:53.328994 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0827aac-46b5-4753-9b41-4958d7ae603a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e527489d37524b42ae9dbc3f8d4f8e7d, user_id=1c4515a7d557456a9fa71105ee5142ef, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:53.331722 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0827aac-46b5-4753-9b41-4958d7ae603a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:53.407559 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1064/2127] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:53 2022] PUT /identity/v3/projects/e527489d37524b42ae9dbc3f8d4f8e7d/users/1c4515a7d557456a9fa71105ee5142ef/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:33:53.411537 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052476f-0462-4dff-a7ea-c478235934e3 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:53.421615 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d052476f-0462-4dff-a7ea-c478235934e3 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:53.480581 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e24136a-a950-4026-a542-971f69541ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:53.481296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24136a-a950-4026-a542-971f69541ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:53.481485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24136a-a950-4026-a542-971f69541ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:53.481753 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e24136a-a950-4026-a542-971f69541ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:53.482421 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e24136a-a950-4026-a542-971f69541ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:53.484039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e24136a-a950-4026-a542-971f69541ed6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:53.494033 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1064/2128] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:53.498124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0827aac-46b5-4753-9b41-4958d7ae603a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:53.509340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0827aac-46b5-4753-9b41-4958d7ae603a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:53.515329 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391aa53d-35b8-436e-b73b-8fdb3fc1af86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:53.516655 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391aa53d-35b8-436e-b73b-8fdb3fc1af86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:53.518860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391aa53d-35b8-436e-b73b-8fdb3fc1af86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:53.518860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391aa53d-35b8-436e-b73b-8fdb3fc1af86 admin admin] PATH_INFO: `/v3/projects/e527489d37524b42ae9dbc3f8d4f8e7d/users/1c4515a7d557456a9fa71105ee5142ef/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:53.520371 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391aa53d-35b8-436e-b73b-8fdb3fc1af86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e527489d37524b42ae9dbc3f8d4f8e7d, user_id=1c4515a7d557456a9fa71105ee5142ef, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:53.523747 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391aa53d-35b8-436e-b73b-8fdb3fc1af86 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:53.536258 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1065/2129] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:53 2022] PUT /identity/v3/projects/e527489d37524b42ae9dbc3f8d4f8e7d/users/1c4515a7d557456a9fa71105ee5142ef/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:53.546836 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb2361-b383-4686-a55b-2c6d5ea2ccb9 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:53.547220 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb2361-b383-4686-a55b-2c6d5ea2ccb9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:53.547502 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb2361-b383-4686-a55b-2c6d5ea2ccb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:53.560284 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391aa53d-35b8-436e-b73b-8fdb3fc1af86 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:53.562572 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-391aa53d-35b8-436e-b73b-8fdb3fc1af86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:53.579117 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-47fb2361-b383-4686-a55b-2c6d5ea2ccb9 None None] MFA Rules not processed for user `1c4515a7d557456a9fa71105ee5142ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:53.583627 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-47fb2361-b383-4686-a55b-2c6d5ea2ccb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:53.636179 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391aa53d-35b8-436e-b73b-8fdb3fc1af86 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:53.640226 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94aa43a-35a3-4837-8a9b-a97901e3ebef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:53.641529 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94aa43a-35a3-4837-8a9b-a97901e3ebef service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:53.641791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94aa43a-35a3-4837-8a9b-a97901e3ebef service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:53.642819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94aa43a-35a3-4837-8a9b-a97901e3ebef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:53.644546 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d94aa43a-35a3-4837-8a9b-a97901e3ebef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:53.675458 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:53.680769 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1065/2130] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:53 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:53.692249 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb2361-b383-4686-a55b-2c6d5ea2ccb9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:53.693817 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-47fb2361-b383-4686-a55b-2c6d5ea2ccb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:53.706994 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb2361-b383-4686-a55b-2c6d5ea2ccb9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:53.710777 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c2bec-aafb-4d9b-8f28-dc5dc171a7c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:53.711448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c2bec-aafb-4d9b-8f28-dc5dc171a7c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:53.711814 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c2bec-aafb-4d9b-8f28-dc5dc171a7c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:53.712150 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c2bec-aafb-4d9b-8f28-dc5dc171a7c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:53.721360 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7c2bec-aafb-4d9b-8f28-dc5dc171a7c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:53.722499 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94aa43a-35a3-4837-8a9b-a97901e3ebef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:53.722879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7c2bec-aafb-4d9b-8f28-dc5dc171a7c0 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:53.723975 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94aa43a-35a3-4837-8a9b-a97901e3ebef service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:53.732375 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1066/2131] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:53 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:33:53.733871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1066/2132] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:53 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:33:53.930098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c2bec-aafb-4d9b-8f28-dc5dc171a7c0 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:53.938905 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c2bec-aafb-4d9b-8f28-dc5dc171a7c0 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:53.942724 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912cfacd-3907-43e0-8de0-c1854a730854 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:53.945188 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912cfacd-3907-43e0-8de0-c1854a730854 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:53.945188 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912cfacd-3907-43e0-8de0-c1854a730854 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:53.945188 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912cfacd-3907-43e0-8de0-c1854a730854 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:53.998100 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:54.001671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912cfacd-3907-43e0-8de0-c1854a730854 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:54.005894 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912cfacd-3907-43e0-8de0-c1854a730854 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:54.012772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1067/2133] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:53 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:33:54.502652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94aa43a-35a3-4837-8a9b-a97901e3ebef service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:54.567230 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94aa43a-35a3-4837-8a9b-a97901e3ebef service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:54.571838 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4869ca-b324-4d5d-b6cc-85ada67f9a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:54.572535 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4869ca-b324-4d5d-b6cc-85ada67f9a73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:54.572834 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4869ca-b324-4d5d-b6cc-85ada67f9a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:54.573111 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4869ca-b324-4d5d-b6cc-85ada67f9a73 admin admin] PATH_INFO: `/v3/users/42b3f8a8dc7b4a6d9c68d41f07035b68` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:54.574277 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4869ca-b324-4d5d-b6cc-85ada67f9a73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42b3f8a8dc7b4a6d9c68d41f07035b68)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:54.576207 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4869ca-b324-4d5d-b6cc-85ada67f9a73 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:54.776401 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0a4869ca-b324-4d5d-b6cc-85ada67f9a73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42b3f8a8dc7b4a6d9c68d41f07035b68'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:54.849181 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0a4869ca-b324-4d5d-b6cc-85ada67f9a73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42b3f8a8dc7b4a6d9c68d41f07035b68'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:54.851761 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0a4869ca-b324-4d5d-b6cc-85ada67f9a73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42b3f8a8dc7b4a6d9c68d41f07035b68'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:54.852386 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0a4869ca-b324-4d5d-b6cc-85ada67f9a73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42b3f8a8dc7b4a6d9c68d41f07035b68'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:54.867927 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0a4869ca-b324-4d5d-b6cc-85ada67f9a73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42b3f8a8dc7b4a6d9c68d41f07035b68'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:54.885195 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1067/2134] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:33:54 2022] DELETE /identity/v3/users/42b3f8a8dc7b4a6d9c68d41f07035b68 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:54.893403 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0db26-bb80-4805-857d-a7b3104387e3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:54.893863 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0db26-bb80-4805-857d-a7b3104387e3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:54.893983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0db26-bb80-4805-857d-a7b3104387e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:54.914587 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-40e0db26-bb80-4805-857d-a7b3104387e3 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:54.915886 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-40e0db26-bb80-4805-857d-a7b3104387e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:54.972625 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1068/2135] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:54.982730 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4869ca-b324-4d5d-b6cc-85ada67f9a73 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:54.984163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0a4869ca-b324-4d5d-b6cc-85ada67f9a73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:55.028510 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:55.042933 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4869ca-b324-4d5d-b6cc-85ada67f9a73 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:55.046786 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56885b68-201e-42b3-ad0a-ede91b22429f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:55.047413 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56885b68-201e-42b3-ad0a-ede91b22429f service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:55.047693 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56885b68-201e-42b3-ad0a-ede91b22429f service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:55.047965 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56885b68-201e-42b3-ad0a-ede91b22429f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:55.057657 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56885b68-201e-42b3-ad0a-ede91b22429f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:55.059068 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56885b68-201e-42b3-ad0a-ede91b22429f service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:55.068612 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1068/2136] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:33:55.547457 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0db26-bb80-4805-857d-a7b3104387e3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:55.549039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-40e0db26-bb80-4805-857d-a7b3104387e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:55.562400 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0db26-bb80-4805-857d-a7b3104387e3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:55.566231 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c50c4-8d48-456a-bde2-bf3652c0915f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:55.566959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c50c4-8d48-456a-bde2-bf3652c0915f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:55.567241 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c50c4-8d48-456a-bde2-bf3652c0915f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:55.567512 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c50c4-8d48-456a-bde2-bf3652c0915f admin admin] PATH_INFO: `/v3/projects/cfd8100a870944c9a2c44523a8791469` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:55.569127 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c50c4-8d48-456a-bde2-bf3652c0915f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfd8100a870944c9a2c44523a8791469)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:55.571098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c50c4-8d48-456a-bde2-bf3652c0915f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:55.679126 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c73c50c4-8d48-456a-bde2-bf3652c0915f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfd8100a870944c9a2c44523a8791469', 'request_id': 'req-c73c50c4-8d48-456a-bde2-bf3652c0915f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:55.757266 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c73c50c4-8d48-456a-bde2-bf3652c0915f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfd8100a870944c9a2c44523a8791469', 'request_id': 'req-c73c50c4-8d48-456a-bde2-bf3652c0915f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:55.763432 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c73c50c4-8d48-456a-bde2-bf3652c0915f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfd8100a870944c9a2c44523a8791469', 'request_id': 'req-c73c50c4-8d48-456a-bde2-bf3652c0915f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:55.775525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c73c50c4-8d48-456a-bde2-bf3652c0915f admin admin] The token cache is being invalidate because project cfd8100a870944c9a2c44523a8791469 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:33:55.775815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c73c50c4-8d48-456a-bde2-bf3652c0915f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:55.776466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c73c50c4-8d48-456a-bde2-bf3652c0915f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:55.777940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1069/2137] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:33:55 2022] DELETE /identity/v3/projects/cfd8100a870944c9a2c44523a8791469 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:55.782072 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56885b68-201e-42b3-ad0a-ede91b22429f service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:55.791651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56885b68-201e-42b3-ad0a-ede91b22429f service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:55.847908 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db12eb8-b361-4214-af2f-9d9f691a424a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:55.848540 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db12eb8-b361-4214-af2f-9d9f691a424a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:55.848815 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db12eb8-b361-4214-af2f-9d9f691a424a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:55.849088 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db12eb8-b361-4214-af2f-9d9f691a424a admin admin] PATH_INFO: `/v3/users/1c4515a7d557456a9fa71105ee5142ef` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:55.850246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db12eb8-b361-4214-af2f-9d9f691a424a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c4515a7d557456a9fa71105ee5142ef)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:55.852230 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db12eb8-b361-4214-af2f-9d9f691a424a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:56.107526 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5db12eb8-b361-4214-af2f-9d9f691a424a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c4515a7d557456a9fa71105ee5142ef'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:56.214482 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5db12eb8-b361-4214-af2f-9d9f691a424a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c4515a7d557456a9fa71105ee5142ef'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:56.215121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5db12eb8-b361-4214-af2f-9d9f691a424a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c4515a7d557456a9fa71105ee5142ef'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:56.215799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5db12eb8-b361-4214-af2f-9d9f691a424a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c4515a7d557456a9fa71105ee5142ef'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:56.228334 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-5db12eb8-b361-4214-af2f-9d9f691a424a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c4515a7d557456a9fa71105ee5142ef'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:56.240313 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1069/2138] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:33:55 2022] DELETE /identity/v3/users/1c4515a7d557456a9fa71105ee5142ef => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:56.615947 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c50c4-8d48-456a-bde2-bf3652c0915f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:56.618688 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c73c50c4-8d48-456a-bde2-bf3652c0915f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:56.640811 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c50c4-8d48-456a-bde2-bf3652c0915f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:56.708587 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d632403c-e7cd-4faa-b365-1004cad17e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:56.709324 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632403c-e7cd-4faa-b365-1004cad17e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:56.709602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632403c-e7cd-4faa-b365-1004cad17e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:56.709872 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632403c-e7cd-4faa-b365-1004cad17e2e admin admin] PATH_INFO: `/v3/projects/e527489d37524b42ae9dbc3f8d4f8e7d` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:56.711523 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d632403c-e7cd-4faa-b365-1004cad17e2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e527489d37524b42ae9dbc3f8d4f8e7d)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:56.713399 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d632403c-e7cd-4faa-b365-1004cad17e2e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:56.736639 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:56.820646 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d632403c-e7cd-4faa-b365-1004cad17e2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e527489d37524b42ae9dbc3f8d4f8e7d', 'request_id': 'req-d632403c-e7cd-4faa-b365-1004cad17e2e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:56.888486 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d632403c-e7cd-4faa-b365-1004cad17e2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e527489d37524b42ae9dbc3f8d4f8e7d', 'request_id': 'req-d632403c-e7cd-4faa-b365-1004cad17e2e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:56.894749 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d632403c-e7cd-4faa-b365-1004cad17e2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e527489d37524b42ae9dbc3f8d4f8e7d', 'request_id': 'req-d632403c-e7cd-4faa-b365-1004cad17e2e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:56.908165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d632403c-e7cd-4faa-b365-1004cad17e2e admin admin] The token cache is being invalidate because project e527489d37524b42ae9dbc3f8d4f8e7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:33:56.908454 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d632403c-e7cd-4faa-b365-1004cad17e2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:56.909120 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d632403c-e7cd-4faa-b365-1004cad17e2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:33:56.910635 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1070/2139] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:33:56 2022] DELETE /identity/v3/projects/e527489d37524b42ae9dbc3f8d4f8e7d => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:56.915440 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27876fa4-cc9d-4efa-bb1e-a5ccf429a6f1 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:56.915909 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27876fa4-cc9d-4efa-bb1e-a5ccf429a6f1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:56.916194 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27876fa4-cc9d-4efa-bb1e-a5ccf429a6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:56.937091 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-27876fa4-cc9d-4efa-bb1e-a5ccf429a6f1 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:56.938517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-27876fa4-cc9d-4efa-bb1e-a5ccf429a6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:56.999223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1070/2140] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:57.008818 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb32ec-4a36-4d9c-af9b-e97ef1147583 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:57.009102 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb32ec-4a36-4d9c-af9b-e97ef1147583 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:57.009379 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb32ec-4a36-4d9c-af9b-e97ef1147583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:57.029731 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6eeb32ec-4a36-4d9c-af9b-e97ef1147583 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:57.030990 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6eeb32ec-4a36-4d9c-af9b-e97ef1147583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:57.039390 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1071/2141] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:57.043649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27876fa4-cc9d-4efa-bb1e-a5ccf429a6f1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:57.048431 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:57.053645 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27876fa4-cc9d-4efa-bb1e-a5ccf429a6f1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:57.059155 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f145484c-673d-4124-8e73-c1c59ebfdf54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:57.059698 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145484c-673d-4124-8e73-c1c59ebfdf54 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:57.059975 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145484c-673d-4124-8e73-c1c59ebfdf54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:57.060246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f145484c-673d-4124-8e73-c1c59ebfdf54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:57.060922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f145484c-673d-4124-8e73-c1c59ebfdf54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:33:57.061222 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f145484c-673d-4124-8e73-c1c59ebfdf54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:57.062861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f145484c-673d-4124-8e73-c1c59ebfdf54 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:57.074257 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1071/2142] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:33:57.078847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeb32ec-4a36-4d9c-af9b-e97ef1147583 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:57.088145 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeb32ec-4a36-4d9c-af9b-e97ef1147583 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:57.092785 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26250957-c557-4d05-899d-36fa191e7c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:57.093377 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26250957-c557-4d05-899d-36fa191e7c14 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:57.093657 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26250957-c557-4d05-899d-36fa191e7c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:57.093929 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26250957-c557-4d05-899d-36fa191e7c14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:57.094562 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26250957-c557-4d05-899d-36fa191e7c14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:57.096489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26250957-c557-4d05-899d-36fa191e7c14 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:57.182258 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1072/2143] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:33:57 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:57.186272 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f145484c-673d-4124-8e73-c1c59ebfdf54 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:57.196423 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f145484c-673d-4124-8e73-c1c59ebfdf54 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:57.255597 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e9ed9-9518-491d-bb98-43c438bf135d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:57.256126 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e9ed9-9518-491d-bb98-43c438bf135d admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:57.256404 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e9ed9-9518-491d-bb98-43c438bf135d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:57.256708 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e9ed9-9518-491d-bb98-43c438bf135d admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:57.257313 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e9ed9-9518-491d-bb98-43c438bf135d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:57.259283 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38e9ed9-9518-491d-bb98-43c438bf135d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:57.352957 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1072/2144] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:33:57 2022] POST /identity/v3/users => generated 288 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:57.357422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26250957-c557-4d05-899d-36fa191e7c14 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:57.368286 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26250957-c557-4d05-899d-36fa191e7c14 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:57.372996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418da24-1780-4dd7-b995-2095754cdf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:57.373591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418da24-1780-4dd7-b995-2095754cdf1c admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:57.373871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418da24-1780-4dd7-b995-2095754cdf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:57.374143 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418da24-1780-4dd7-b995-2095754cdf1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:57.374810 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4418da24-1780-4dd7-b995-2095754cdf1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:57.376401 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4418da24-1780-4dd7-b995-2095754cdf1c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:57.386412 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1073/2145] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:57.390404 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e9ed9-9518-491d-bb98-43c438bf135d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:57.400825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e9ed9-9518-491d-bb98-43c438bf135d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:57.405549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354b769-5e95-4111-b8f3-082aa032a96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:57.406144 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354b769-5e95-4111-b8f3-082aa032a96c admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:57.406427 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354b769-5e95-4111-b8f3-082aa032a96c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:57.406790 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354b769-5e95-4111-b8f3-082aa032a96c admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:57.407377 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354b769-5e95-4111-b8f3-082aa032a96c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:57.408956 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354b769-5e95-4111-b8f3-082aa032a96c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:57.419917 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1073/2146] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:57.424079 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418da24-1780-4dd7-b995-2095754cdf1c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:57.433666 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418da24-1780-4dd7-b995-2095754cdf1c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:57.438193 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d300933-e391-40ed-9c65-c41f576caa34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:57.438875 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d300933-e391-40ed-9c65-c41f576caa34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:57.439156 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d300933-e391-40ed-9c65-c41f576caa34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:57.439431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d300933-e391-40ed-9c65-c41f576caa34 admin admin] PATH_INFO: `/v3/projects/058dcd9fc32c41aab0b701e31ba4e90e/users/7a0a6b62301e40308b0c3a4c726421b5/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:57.454499 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d300933-e391-40ed-9c65-c41f576caa34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=058dcd9fc32c41aab0b701e31ba4e90e, user_id=7a0a6b62301e40308b0c3a4c726421b5, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:57.457092 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d300933-e391-40ed-9c65-c41f576caa34 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:57.540613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1074/2147] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:57 2022] PUT /identity/v3/projects/058dcd9fc32c41aab0b701e31ba4e90e/users/7a0a6b62301e40308b0c3a4c726421b5/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:57.550613 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860b7ae-3b5b-44e2-b5e0-00104fb20c3d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:57.550949 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860b7ae-3b5b-44e2-b5e0-00104fb20c3d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:57.551228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860b7ae-3b5b-44e2-b5e0-00104fb20c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:57.580416 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c860b7ae-3b5b-44e2-b5e0-00104fb20c3d None None] MFA Rules not processed for user `7a0a6b62301e40308b0c3a4c726421b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:57.581682 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c860b7ae-3b5b-44e2-b5e0-00104fb20c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:57.644287 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1074/2148] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:33:57.649425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d300933-e391-40ed-9c65-c41f576caa34 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:57.719831 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d300933-e391-40ed-9c65-c41f576caa34 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:57.724505 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b4a5cc-c6e2-4533-baee-73d868f52ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:57.725110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4a5cc-c6e2-4533-baee-73d868f52ceb admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:57.725389 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4a5cc-c6e2-4533-baee-73d868f52ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:57.726324 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4a5cc-c6e2-4533-baee-73d868f52ceb admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:57.727000 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b4a5cc-c6e2-4533-baee-73d868f52ceb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:57.728892 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b4a5cc-c6e2-4533-baee-73d868f52ceb admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:57.812326 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1075/2149] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:33:57 2022] POST /identity/v3/projects => generated 375 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:57.815573 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860b7ae-3b5b-44e2-b5e0-00104fb20c3d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:57.849930 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860b7ae-3b5b-44e2-b5e0-00104fb20c3d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:57.905121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:57.910990 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0711e6-d920-4144-93f0-da1e0c3f1869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:57.911646 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0711e6-d920-4144-93f0-da1e0c3f1869 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:57.911924 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0711e6-d920-4144-93f0-da1e0c3f1869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:57.912195 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0711e6-d920-4144-93f0-da1e0c3f1869 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:57.912817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0711e6-d920-4144-93f0-da1e0c3f1869 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:57.914865 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0711e6-d920-4144-93f0-da1e0c3f1869 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:58.000309 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1075/2150] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:33:57 2022] POST /identity/v3/users => generated 294 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:33:58.001523 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b4a5cc-c6e2-4533-baee-73d868f52ceb admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:58.012723 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b4a5cc-c6e2-4533-baee-73d868f52ceb admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:58.017487 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66715b69-2430-4717-b3bb-ce015441d52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:58.018106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66715b69-2430-4717-b3bb-ce015441d52d admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:58.018424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66715b69-2430-4717-b3bb-ce015441d52d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:58.018689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66715b69-2430-4717-b3bb-ce015441d52d admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:58.019358 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66715b69-2430-4717-b3bb-ce015441d52d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:58.020936 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66715b69-2430-4717-b3bb-ce015441d52d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:58.032591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1076/2151] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:33:58.036685 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0711e6-d920-4144-93f0-da1e0c3f1869 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:58.046473 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0711e6-d920-4144-93f0-da1e0c3f1869 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:58.050999 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a387bd-8a5e-432c-87e6-74e27b4efee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:58.051640 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a387bd-8a5e-432c-87e6-74e27b4efee8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:58.051917 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a387bd-8a5e-432c-87e6-74e27b4efee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:58.052187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a387bd-8a5e-432c-87e6-74e27b4efee8 admin admin] PATH_INFO: `/v3/projects/f3967467d9554bc1a717602e7034c79f/users/769889a9412b4851add6e10baf386941/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:58.066863 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a387bd-8a5e-432c-87e6-74e27b4efee8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3967467d9554bc1a717602e7034c79f, user_id=769889a9412b4851add6e10baf386941, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:58.070126 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a387bd-8a5e-432c-87e6-74e27b4efee8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:58.139373 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1076/2152] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:58 2022] PUT /identity/v3/projects/f3967467d9554bc1a717602e7034c79f/users/769889a9412b4851add6e10baf386941/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:33:58.143369 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66715b69-2430-4717-b3bb-ce015441d52d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:58.153924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66715b69-2430-4717-b3bb-ce015441d52d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:58.168744 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:33:58.213256 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bb857b-9068-4616-87c7-6778a7413cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:58.213865 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb857b-9068-4616-87c7-6778a7413cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:58.214145 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb857b-9068-4616-87c7-6778a7413cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:58.214415 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bb857b-9068-4616-87c7-6778a7413cd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:58.220202 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb857b-9068-4616-87c7-6778a7413cd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:58.220202 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bb857b-9068-4616-87c7-6778a7413cd4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:58.226725 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1077/2153] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:33:58.231025 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a387bd-8a5e-432c-87e6-74e27b4efee8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:58.242198 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a387bd-8a5e-432c-87e6-74e27b4efee8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:58.247008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4cf2ad-4172-4d34-92ac-9886fb0ffe2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:58.247669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cf2ad-4172-4d34-92ac-9886fb0ffe2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:58.247941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cf2ad-4172-4d34-92ac-9886fb0ffe2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:58.248214 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4cf2ad-4172-4d34-92ac-9886fb0ffe2d admin admin] PATH_INFO: `/v3/projects/f3967467d9554bc1a717602e7034c79f/users/769889a9412b4851add6e10baf386941/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:58.250126 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4cf2ad-4172-4d34-92ac-9886fb0ffe2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3967467d9554bc1a717602e7034c79f, user_id=769889a9412b4851add6e10baf386941, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:58.253013 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4cf2ad-4172-4d34-92ac-9886fb0ffe2d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:58.264031 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1077/2154] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:33:58 2022] PUT /identity/v3/projects/f3967467d9554bc1a717602e7034c79f/users/769889a9412b4851add6e10baf386941/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:33:58.275112 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91201364-8487-44c4-83a3-db0c78566a2f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:58.275391 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91201364-8487-44c4-83a3-db0c78566a2f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:58.275672 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91201364-8487-44c4-83a3-db0c78566a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:58.306034 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-91201364-8487-44c4-83a3-db0c78566a2f None None] MFA Rules not processed for user `769889a9412b4851add6e10baf386941`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:33:58.307431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-91201364-8487-44c4-83a3-db0c78566a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:58.378353 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1078/2155] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:33:58 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:33:58.391289 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4cf2ad-4172-4d34-92ac-9886fb0ffe2d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:33:58.391474 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6b4cf2ad-4172-4d34-92ac-9886fb0ffe2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:33:58.448407 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4cf2ad-4172-4d34-92ac-9886fb0ffe2d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:33:58.452353 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe9523-ed6d-4b85-ab23-b40345fdfa03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:33:58.452961 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe9523-ed6d-4b85-ab23-b40345fdfa03 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:33:58.453236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe9523-ed6d-4b85-ab23-b40345fdfa03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:33:58.453504 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe9523-ed6d-4b85-ab23-b40345fdfa03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:33:58.510895 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfe9523-ed6d-4b85-ab23-b40345fdfa03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:33:58.512713 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfe9523-ed6d-4b85-ab23-b40345fdfa03 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:33:58.524068 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1078/2156] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:33:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:34:01.018827 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91201364-8487-44c4-83a3-db0c78566a2f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:01.020282 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-91201364-8487-44c4-83a3-db0c78566a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:01.113409 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91201364-8487-44c4-83a3-db0c78566a2f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:01.118064 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2b6ca-1fd5-4989-9cf2-8737ec2d979f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:01.118728 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2b6ca-1fd5-4989-9cf2-8737ec2d979f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:01.119009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2b6ca-1fd5-4989-9cf2-8737ec2d979f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:01.119286 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2b6ca-1fd5-4989-9cf2-8737ec2d979f admin admin] PATH_INFO: `/v3/users/445190e0a56e4b409de1333d9e915019` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:01.120428 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b2b6ca-1fd5-4989-9cf2-8737ec2d979f admin admin] RBAC: Authorizing `identity:delete_user(user_id=445190e0a56e4b409de1333d9e915019)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:01.122348 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b2b6ca-1fd5-4989-9cf2-8737ec2d979f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:01.377447 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:01.382955 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-97b2b6ca-1fd5-4989-9cf2-8737ec2d979f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '445190e0a56e4b409de1333d9e915019'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:01.390124 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-97b2b6ca-1fd5-4989-9cf2-8737ec2d979f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '445190e0a56e4b409de1333d9e915019'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:01.463776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-97b2b6ca-1fd5-4989-9cf2-8737ec2d979f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '445190e0a56e4b409de1333d9e915019'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:01.464484 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-97b2b6ca-1fd5-4989-9cf2-8737ec2d979f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '445190e0a56e4b409de1333d9e915019'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:01.477141 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-97b2b6ca-1fd5-4989-9cf2-8737ec2d979f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '445190e0a56e4b409de1333d9e915019'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:01.483878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1079/2157] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:34:01 2022] DELETE /identity/v3/users/445190e0a56e4b409de1333d9e915019 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:01.781733 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe9523-ed6d-4b85-ab23-b40345fdfa03 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:01.804954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe9523-ed6d-4b85-ab23-b40345fdfa03 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:01.887879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a7275-2f7d-45f9-b353-b5f70a8c64df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:01.888435 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a7275-2f7d-45f9-b353-b5f70a8c64df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:01.888713 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a7275-2f7d-45f9-b353-b5f70a8c64df admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:01.888980 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a7275-2f7d-45f9-b353-b5f70a8c64df admin admin] PATH_INFO: `/v3/projects/5e3423cc79d04e83a7e0e292ac973188` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:01.890107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a7275-2f7d-45f9-b353-b5f70a8c64df admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e3423cc79d04e83a7e0e292ac973188)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:01.892037 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a7275-2f7d-45f9-b353-b5f70a8c64df admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:01.898852 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:02.026483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-875a7275-2f7d-45f9-b353-b5f70a8c64df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e3423cc79d04e83a7e0e292ac973188', 'request_id': 'req-875a7275-2f7d-45f9-b353-b5f70a8c64df'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:02.032232 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-875a7275-2f7d-45f9-b353-b5f70a8c64df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e3423cc79d04e83a7e0e292ac973188', 'request_id': 'req-875a7275-2f7d-45f9-b353-b5f70a8c64df'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:02.039100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-875a7275-2f7d-45f9-b353-b5f70a8c64df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e3423cc79d04e83a7e0e292ac973188', 'request_id': 'req-875a7275-2f7d-45f9-b353-b5f70a8c64df'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:02.111197 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-875a7275-2f7d-45f9-b353-b5f70a8c64df admin admin] The token cache is being invalidate because project 5e3423cc79d04e83a7e0e292ac973188 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:34:02.111508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-875a7275-2f7d-45f9-b353-b5f70a8c64df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:02.112189 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-875a7275-2f7d-45f9-b353-b5f70a8c64df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:02.115261 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1079/2158] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:34:01 2022] DELETE /identity/v3/projects/5e3423cc79d04e83a7e0e292ac973188 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:02.119276 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2b6ca-1fd5-4989-9cf2-8737ec2d979f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:02.120775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-97b2b6ca-1fd5-4989-9cf2-8737ec2d979f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:02.134698 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2b6ca-1fd5-4989-9cf2-8737ec2d979f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:02.204083 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c438e6-c965-4523-a2b3-7ab21daf572f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:02.204720 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c438e6-c965-4523-a2b3-7ab21daf572f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:02.205062 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c438e6-c965-4523-a2b3-7ab21daf572f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:02.205400 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c438e6-c965-4523-a2b3-7ab21daf572f admin admin] PATH_INFO: `/v3/users/fc54203f08f34a4dabf2ff1aeef8490c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:02.206634 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c438e6-c965-4523-a2b3-7ab21daf572f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc54203f08f34a4dabf2ff1aeef8490c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:02.208617 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c438e6-c965-4523-a2b3-7ab21daf572f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:02.449877 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-02c438e6-c965-4523-a2b3-7ab21daf572f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc54203f08f34a4dabf2ff1aeef8490c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:02.461822 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-02c438e6-c965-4523-a2b3-7ab21daf572f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc54203f08f34a4dabf2ff1aeef8490c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:02.573773 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-02c438e6-c965-4523-a2b3-7ab21daf572f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc54203f08f34a4dabf2ff1aeef8490c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:02.574419 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-02c438e6-c965-4523-a2b3-7ab21daf572f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc54203f08f34a4dabf2ff1aeef8490c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:02.589179 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-02c438e6-c965-4523-a2b3-7ab21daf572f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc54203f08f34a4dabf2ff1aeef8490c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:02.595430 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1080/2159] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:34:02 2022] DELETE /identity/v3/users/fc54203f08f34a4dabf2ff1aeef8490c => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:02.906446 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a7275-2f7d-45f9-b353-b5f70a8c64df admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:02.907897 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-875a7275-2f7d-45f9-b353-b5f70a8c64df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:02.929856 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a7275-2f7d-45f9-b353-b5f70a8c64df admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:02.995178 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12514963-d4b1-42f3-a9ed-6df4f9643f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:02.995965 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12514963-d4b1-42f3-a9ed-6df4f9643f83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:02.996090 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12514963-d4b1-42f3-a9ed-6df4f9643f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:02.996543 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12514963-d4b1-42f3-a9ed-6df4f9643f83 admin admin] PATH_INFO: `/v3/projects/cd7bf0a45e214c0f93ad66e1f8dc68ec` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:02.997775 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12514963-d4b1-42f3-a9ed-6df4f9643f83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd7bf0a45e214c0f93ad66e1f8dc68ec)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:02.999738 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12514963-d4b1-42f3-a9ed-6df4f9643f83 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:02.999924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c438e6-c965-4523-a2b3-7ab21daf572f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:03.001428 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-02c438e6-c965-4523-a2b3-7ab21daf572f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:03.020014 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c438e6-c965-4523-a2b3-7ab21daf572f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:03.026018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603dd981-9508-4443-8cb9-a23f4d72036c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:03.026881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603dd981-9508-4443-8cb9-a23f4d72036c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:03.027295 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603dd981-9508-4443-8cb9-a23f4d72036c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:03.034237 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603dd981-9508-4443-8cb9-a23f4d72036c admin admin] PATH_INFO: `/v3/users/7a0a6b62301e40308b0c3a4c726421b5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:03.035739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603dd981-9508-4443-8cb9-a23f4d72036c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a0a6b62301e40308b0c3a4c726421b5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:03.038257 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603dd981-9508-4443-8cb9-a23f4d72036c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:03.115778 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-12514963-d4b1-42f3-a9ed-6df4f9643f83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd7bf0a45e214c0f93ad66e1f8dc68ec', 'request_id': 'req-12514963-d4b1-42f3-a9ed-6df4f9643f83'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:03.122486 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-12514963-d4b1-42f3-a9ed-6df4f9643f83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd7bf0a45e214c0f93ad66e1f8dc68ec', 'request_id': 'req-12514963-d4b1-42f3-a9ed-6df4f9643f83'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:03.129237 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-12514963-d4b1-42f3-a9ed-6df4f9643f83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd7bf0a45e214c0f93ad66e1f8dc68ec', 'request_id': 'req-12514963-d4b1-42f3-a9ed-6df4f9643f83'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:03.265575 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:03.282189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-603dd981-9508-4443-8cb9-a23f4d72036c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a0a6b62301e40308b0c3a4c726421b5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:03.283185 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-603dd981-9508-4443-8cb9-a23f4d72036c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a0a6b62301e40308b0c3a4c726421b5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:03.285118 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-12514963-d4b1-42f3-a9ed-6df4f9643f83 admin admin] The token cache is being invalidate because project cd7bf0a45e214c0f93ad66e1f8dc68ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:34:03.285489 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-12514963-d4b1-42f3-a9ed-6df4f9643f83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:03.286208 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-12514963-d4b1-42f3-a9ed-6df4f9643f83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:03.289191 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1080/2160] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:34:02 2022] DELETE /identity/v3/projects/cd7bf0a45e214c0f93ad66e1f8dc68ec => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:03.299056 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637edeee-c66c-4b7f-ad5f-3b2057c3eda1 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:03.299348 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637edeee-c66c-4b7f-ad5f-3b2057c3eda1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:03.299626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637edeee-c66c-4b7f-ad5f-3b2057c3eda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:03.320201 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-637edeee-c66c-4b7f-ad5f-3b2057c3eda1 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:03.321419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-637edeee-c66c-4b7f-ad5f-3b2057c3eda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:03.340957 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:03.387290 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1081/2161] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:34:03.395660 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd084e6-deed-420d-a6eb-b59607152339 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:03.395942 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd084e6-deed-420d-a6eb-b59607152339 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:03.396218 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd084e6-deed-420d-a6eb-b59607152339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:03.419500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9fd084e6-deed-420d-a6eb-b59607152339 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:03.420316 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-603dd981-9508-4443-8cb9-a23f4d72036c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a0a6b62301e40308b0c3a4c726421b5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:03.420450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9fd084e6-deed-420d-a6eb-b59607152339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:03.421457 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-603dd981-9508-4443-8cb9-a23f4d72036c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a0a6b62301e40308b0c3a4c726421b5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:03.430640 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1082/2162] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:03.434639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd084e6-deed-420d-a6eb-b59607152339 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:03.440560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-603dd981-9508-4443-8cb9-a23f4d72036c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a0a6b62301e40308b0c3a4c726421b5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:03.444383 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd084e6-deed-420d-a6eb-b59607152339 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:03.449885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1dd277-a40c-4ac8-83d3-605da980cee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:03.450502 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1dd277-a40c-4ac8-83d3-605da980cee6 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:03.450817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1dd277-a40c-4ac8-83d3-605da980cee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:03.451106 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1dd277-a40c-4ac8-83d3-605da980cee6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:03.451942 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1dd277-a40c-4ac8-83d3-605da980cee6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:34:03.452182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1dd277-a40c-4ac8-83d3-605da980cee6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:03.453455 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1081/2163] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:34:02 2022] DELETE /identity/v3/users/7a0a6b62301e40308b0c3a4c726421b5 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:34:03.453849 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1dd277-a40c-4ac8-83d3-605da980cee6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:03.462320 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff0c38-2103-4f92-942e-56bac9694a0f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:03.462803 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff0c38-2103-4f92-942e-56bac9694a0f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:03.463569 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff0c38-2103-4f92-942e-56bac9694a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:03.467487 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1083/2164] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:34:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:34:03.471801 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1dd277-a40c-4ac8-83d3-605da980cee6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:03.485142 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1dd277-a40c-4ac8-83d3-605da980cee6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:03.509539 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a9ff0c38-2103-4f92-942e-56bac9694a0f None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:03.515045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a9ff0c38-2103-4f92-942e-56bac9694a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:03.567694 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8946e-2a87-4732-9d3f-e3e5b9c035c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:03.619979 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8946e-2a87-4732-9d3f-e3e5b9c035c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:03.619979 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8946e-2a87-4732-9d3f-e3e5b9c035c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:03.619979 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8946e-2a87-4732-9d3f-e3e5b9c035c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:03.619979 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc8946e-2a87-4732-9d3f-e3e5b9c035c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:03.619979 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc8946e-2a87-4732-9d3f-e3e5b9c035c8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:03.642490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1082/2165] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:03.678944 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff0c38-2103-4f92-942e-56bac9694a0f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:03.681914 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a9ff0c38-2103-4f92-942e-56bac9694a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:03.759767 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff0c38-2103-4f92-942e-56bac9694a0f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:03.763619 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfc739-1e54-4b2e-8783-d25fa37d5ae3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:03.764233 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfc739-1e54-4b2e-8783-d25fa37d5ae3 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:03.764512 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfc739-1e54-4b2e-8783-d25fa37d5ae3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:03.764785 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcfc739-1e54-4b2e-8783-d25fa37d5ae3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:03.776280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcfc739-1e54-4b2e-8783-d25fa37d5ae3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:03.777978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcfc739-1e54-4b2e-8783-d25fa37d5ae3 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:03.790900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1083/2166] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:34:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:34:03.805701 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1084/2167] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:34:03 2022] POST /identity/v3/projects => generated 375 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:34:03.809665 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfc739-1e54-4b2e-8783-d25fa37d5ae3 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:03.825631 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcfc739-1e54-4b2e-8783-d25fa37d5ae3 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:03.886261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:03.895031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8141f-7e11-46d0-9bd0-913d69640409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:03.895650 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8141f-7e11-46d0-9bd0-913d69640409 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:03.895937 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8141f-7e11-46d0-9bd0-913d69640409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:03.896239 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8141f-7e11-46d0-9bd0-913d69640409 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:03.896900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be8141f-7e11-46d0-9bd0-913d69640409 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:03.899434 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be8141f-7e11-46d0-9bd0-913d69640409 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:04.017555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8946e-2a87-4732-9d3f-e3e5b9c035c8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:04.019950 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7fc8946e-2a87-4732-9d3f-e3e5b9c035c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:04.033153 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8946e-2a87-4732-9d3f-e3e5b9c035c8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:04.039609 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1084/2168] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:34:03 2022] POST /identity/v3/users => generated 288 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:34:04.041253 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fbbce9-0a81-4bc3-aa81-42914b985ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:04.041890 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbbce9-0a81-4bc3-aa81-42914b985ffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:04.042170 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbbce9-0a81-4bc3-aa81-42914b985ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:04.043354 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbbce9-0a81-4bc3-aa81-42914b985ffc admin admin] PATH_INFO: `/v3/users/ceb389101cdc4f8db5a69d0c653fc15e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:04.043915 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8141f-7e11-46d0-9bd0-913d69640409 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:04.044043 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fbbce9-0a81-4bc3-aa81-42914b985ffc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ceb389101cdc4f8db5a69d0c653fc15e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:04.045974 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fbbce9-0a81-4bc3-aa81-42914b985ffc admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:04.055389 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8141f-7e11-46d0-9bd0-913d69640409 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:04.064818 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6d842-7c5a-42dc-961e-e538d1dae4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:04.067186 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6d842-7c5a-42dc-961e-e538d1dae4cd admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:04.069972 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6d842-7c5a-42dc-961e-e538d1dae4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:04.069972 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6d842-7c5a-42dc-961e-e538d1dae4cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:04.073350 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6d842-7c5a-42dc-961e-e538d1dae4cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:04.073806 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6d842-7c5a-42dc-961e-e538d1dae4cd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:04.089319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1085/2169] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:34:04.090448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6d842-7c5a-42dc-961e-e538d1dae4cd admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:04.101718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6d842-7c5a-42dc-961e-e538d1dae4cd admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:04.110849 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1db07a-e733-407d-ba1f-10ea9317b632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:04.112171 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1db07a-e733-407d-ba1f-10ea9317b632 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:04.112458 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1db07a-e733-407d-ba1f-10ea9317b632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:04.112884 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1db07a-e733-407d-ba1f-10ea9317b632 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:04.113552 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1db07a-e733-407d-ba1f-10ea9317b632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:04.115935 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1db07a-e733-407d-ba1f-10ea9317b632 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:04.128815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1086/2170] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:34:04.133418 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1db07a-e733-407d-ba1f-10ea9317b632 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:04.144064 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1db07a-e733-407d-ba1f-10ea9317b632 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:04.150069 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5d552b-f93a-4c6e-baa7-1140bf8a4370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:04.157042 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d552b-f93a-4c6e-baa7-1140bf8a4370 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:04.157676 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d552b-f93a-4c6e-baa7-1140bf8a4370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:04.158151 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5d552b-f93a-4c6e-baa7-1140bf8a4370 admin admin] PATH_INFO: `/v3/projects/633915e61f6e4acaa0de4ff3861d8526/users/a4b8c1479d3d4905b777f060a76c5cb2/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:04.183680 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d552b-f93a-4c6e-baa7-1140bf8a4370 admin admin] RBAC: Authorizing `identity:create_grant(project_id=633915e61f6e4acaa0de4ff3861d8526, user_id=a4b8c1479d3d4905b777f060a76c5cb2, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:04.186669 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5d552b-f93a-4c6e-baa7-1140bf8a4370 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:04.271219 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-33fbbce9-0a81-4bc3-aa81-42914b985ffc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ceb389101cdc4f8db5a69d0c653fc15e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:04.351241 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1087/2171] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:34:04 2022] PUT /identity/v3/projects/633915e61f6e4acaa0de4ff3861d8526/users/a4b8c1479d3d4905b777f060a76c5cb2/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:34:04.364700 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d010be-ba37-4bad-821b-dde855c4cb38 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:04.365013 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d010be-ba37-4bad-821b-dde855c4cb38 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:04.365294 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d010be-ba37-4bad-821b-dde855c4cb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:04.374639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-33fbbce9-0a81-4bc3-aa81-42914b985ffc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ceb389101cdc4f8db5a69d0c653fc15e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:04.377405 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-33fbbce9-0a81-4bc3-aa81-42914b985ffc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ceb389101cdc4f8db5a69d0c653fc15e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:04.378840 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-33fbbce9-0a81-4bc3-aa81-42914b985ffc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ceb389101cdc4f8db5a69d0c653fc15e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:04.395892 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-33fbbce9-0a81-4bc3-aa81-42914b985ffc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ceb389101cdc4f8db5a69d0c653fc15e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:04.399485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-10d010be-ba37-4bad-821b-dde855c4cb38 None None] MFA Rules not processed for user `a4b8c1479d3d4905b777f060a76c5cb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:04.401309 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-10d010be-ba37-4bad-821b-dde855c4cb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:04.413978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1085/2172] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:34:04 2022] DELETE /identity/v3/users/ceb389101cdc4f8db5a69d0c653fc15e => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:34:04.415684 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fbbce9-0a81-4bc3-aa81-42914b985ffc admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:04.417393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-33fbbce9-0a81-4bc3-aa81-42914b985ffc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:04.425771 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:04.498339 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1088/2173] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:34:04.499552 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cbc59-f3e0-473c-9489-64042a5f0d3e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:04.499835 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cbc59-f3e0-473c-9489-64042a5f0d3e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:04.500251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cbc59-f3e0-473c-9489-64042a5f0d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:04.502409 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fbbce9-0a81-4bc3-aa81-42914b985ffc admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:04.507907 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcfdf3e-cf2b-4061-87fa-acf697f35f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:04.508552 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfdf3e-cf2b-4061-87fa-acf697f35f92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:04.508895 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfdf3e-cf2b-4061-87fa-acf697f35f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:04.509251 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcfdf3e-cf2b-4061-87fa-acf697f35f92 admin admin] PATH_INFO: `/v3/projects/058dcd9fc32c41aab0b701e31ba4e90e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:04.510373 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcfdf3e-cf2b-4061-87fa-acf697f35f92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=058dcd9fc32c41aab0b701e31ba4e90e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:04.512301 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcfdf3e-cf2b-4061-87fa-acf697f35f92 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:04.522764 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4d8cbc59-f3e0-473c-9489-64042a5f0d3e None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:04.523973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4d8cbc59-f3e0-473c-9489-64042a5f0d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:04.531618 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1089/2174] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:04.532506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8cbc59-f3e0-473c-9489-64042a5f0d3e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:04.534156 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4d8cbc59-f3e0-473c-9489-64042a5f0d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:04.593216 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8cbc59-f3e0-473c-9489-64042a5f0d3e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:04.596880 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d0464-b877-4633-b345-3439428db78d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:04.597491 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d0464-b877-4633-b345-3439428db78d service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:04.597776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d0464-b877-4633-b345-3439428db78d service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:04.598047 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d0464-b877-4633-b345-3439428db78d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:04.656118 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d0464-b877-4633-b345-3439428db78d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:04.657594 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d0464-b877-4633-b345-3439428db78d service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:04.667641 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1090/2175] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:34:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:34:04.668524 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d0464-b877-4633-b345-3439428db78d service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:04.670035 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f85d0464-b877-4633-b345-3439428db78d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:04.687275 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:04.687794 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4fcfdf3e-cf2b-4061-87fa-acf697f35f92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '058dcd9fc32c41aab0b701e31ba4e90e', 'request_id': 'req-4fcfdf3e-cf2b-4061-87fa-acf697f35f92'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:04.694017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4fcfdf3e-cf2b-4061-87fa-acf697f35f92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '058dcd9fc32c41aab0b701e31ba4e90e', 'request_id': 'req-4fcfdf3e-cf2b-4061-87fa-acf697f35f92'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:04.700342 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4fcfdf3e-cf2b-4061-87fa-acf697f35f92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '058dcd9fc32c41aab0b701e31ba4e90e', 'request_id': 'req-4fcfdf3e-cf2b-4061-87fa-acf697f35f92'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:04.737472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d0464-b877-4633-b345-3439428db78d service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:04.782034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4fcfdf3e-cf2b-4061-87fa-acf697f35f92 admin admin] The token cache is being invalidate because project 058dcd9fc32c41aab0b701e31ba4e90e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:34:04.782679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4fcfdf3e-cf2b-4061-87fa-acf697f35f92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:04.784084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4fcfdf3e-cf2b-4061-87fa-acf697f35f92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:04.787154 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1086/2176] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:34:04 2022] DELETE /identity/v3/projects/058dcd9fc32c41aab0b701e31ba4e90e => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:04.791262 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcfdf3e-cf2b-4061-87fa-acf697f35f92 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:04.793291 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4fcfdf3e-cf2b-4061-87fa-acf697f35f92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:04.833770 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003a936-9936-4369-8aac-4f53c0c315a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:04.834581 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003a936-9936-4369-8aac-4f53c0c315a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:04.835804 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003a936-9936-4369-8aac-4f53c0c315a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:04.836178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003a936-9936-4369-8aac-4f53c0c315a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:04.846224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c003a936-9936-4369-8aac-4f53c0c315a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:04.930549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcfdf3e-cf2b-4061-87fa-acf697f35f92 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:04.937273 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f96a196-7398-454c-bd41-af4beaa3f40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:04.937994 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96a196-7398-454c-bd41-af4beaa3f40d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:04.938367 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96a196-7398-454c-bd41-af4beaa3f40d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:04.938939 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96a196-7398-454c-bd41-af4beaa3f40d admin admin] PATH_INFO: `/v3/users/769889a9412b4851add6e10baf386941` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:04.940500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f96a196-7398-454c-bd41-af4beaa3f40d admin admin] RBAC: Authorizing `identity:delete_user(user_id=769889a9412b4851add6e10baf386941)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:04.943766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f96a196-7398-454c-bd41-af4beaa3f40d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:04.951356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003a936-9936-4369-8aac-4f53c0c315a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:04.952879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003a936-9936-4369-8aac-4f53c0c315a7 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:04.962748 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1091/2177] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:34:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:34:05.143993 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:05.153636 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2f96a196-7398-454c-bd41-af4beaa3f40d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '769889a9412b4851add6e10baf386941'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:05.268988 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2f96a196-7398-454c-bd41-af4beaa3f40d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '769889a9412b4851add6e10baf386941'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:05.269609 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2f96a196-7398-454c-bd41-af4beaa3f40d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '769889a9412b4851add6e10baf386941'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:05.270319 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2f96a196-7398-454c-bd41-af4beaa3f40d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '769889a9412b4851add6e10baf386941'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:05.289556 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2f96a196-7398-454c-bd41-af4beaa3f40d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '769889a9412b4851add6e10baf386941'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:05.305059 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1087/2178] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:34:04 2022] DELETE /identity/v3/users/769889a9412b4851add6e10baf386941 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:05.482672 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003a936-9936-4369-8aac-4f53c0c315a7 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:05.484300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c003a936-9936-4369-8aac-4f53c0c315a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:05.529097 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003a936-9936-4369-8aac-4f53c0c315a7 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:05.601500 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:05.622791 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76126b0c-8af7-47c9-9079-98b3f21d35b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:05.623399 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76126b0c-8af7-47c9-9079-98b3f21d35b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:05.623685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76126b0c-8af7-47c9-9079-98b3f21d35b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:05.623958 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76126b0c-8af7-47c9-9079-98b3f21d35b2 admin admin] PATH_INFO: `/v3/projects/28efe27ecdf04c4581e855ddd7f07bc5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:05.625105 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76126b0c-8af7-47c9-9079-98b3f21d35b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28efe27ecdf04c4581e855ddd7f07bc5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:05.627478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76126b0c-8af7-47c9-9079-98b3f21d35b2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:05.792305 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-76126b0c-8af7-47c9-9079-98b3f21d35b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28efe27ecdf04c4581e855ddd7f07bc5', 'request_id': 'req-76126b0c-8af7-47c9-9079-98b3f21d35b2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:05.913487 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-76126b0c-8af7-47c9-9079-98b3f21d35b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28efe27ecdf04c4581e855ddd7f07bc5', 'request_id': 'req-76126b0c-8af7-47c9-9079-98b3f21d35b2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:05.913487 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-76126b0c-8af7-47c9-9079-98b3f21d35b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28efe27ecdf04c4581e855ddd7f07bc5', 'request_id': 'req-76126b0c-8af7-47c9-9079-98b3f21d35b2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:05.918936 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f96a196-7398-454c-bd41-af4beaa3f40d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:05.920572 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2f96a196-7398-454c-bd41-af4beaa3f40d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:05.941992 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-76126b0c-8af7-47c9-9079-98b3f21d35b2 admin admin] The token cache is being invalidate because project 28efe27ecdf04c4581e855ddd7f07bc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:34:05.941992 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-76126b0c-8af7-47c9-9079-98b3f21d35b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:05.941992 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-76126b0c-8af7-47c9-9079-98b3f21d35b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:05.941992 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1092/2179] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:34:05 2022] DELETE /identity/v3/projects/28efe27ecdf04c4581e855ddd7f07bc5 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:05.946550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76126b0c-8af7-47c9-9079-98b3f21d35b2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:05.951368 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-76126b0c-8af7-47c9-9079-98b3f21d35b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:05.953384 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f96a196-7398-454c-bd41-af4beaa3f40d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:06.037716 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76126b0c-8af7-47c9-9079-98b3f21d35b2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:06.038288 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5dbca-952f-49de-bea4-6d1ceb7efb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:06.039096 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5dbca-952f-49de-bea4-6d1ceb7efb90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:06.039465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5dbca-952f-49de-bea4-6d1ceb7efb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:06.039761 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5dbca-952f-49de-bea4-6d1ceb7efb90 admin admin] PATH_INFO: `/v3/projects/f3967467d9554bc1a717602e7034c79f` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:06.041026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef5dbca-952f-49de-bea4-6d1ceb7efb90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3967467d9554bc1a717602e7034c79f)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:06.043118 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef5dbca-952f-49de-bea4-6d1ceb7efb90 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:06.043351 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c9b89c-9689-4c3d-8fa6-18f2ef6dc3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:06.046727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9b89c-9689-4c3d-8fa6-18f2ef6dc3ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:06.046727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9b89c-9689-4c3d-8fa6-18f2ef6dc3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:06.046727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c9b89c-9689-4c3d-8fa6-18f2ef6dc3ce admin admin] PATH_INFO: `/v3/users/67951aee09b34949aa5971136e45b895` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:06.046727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c9b89c-9689-4c3d-8fa6-18f2ef6dc3ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=67951aee09b34949aa5971136e45b895)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:06.047844 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c9b89c-9689-4c3d-8fa6-18f2ef6dc3ce admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:06.384315 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-95c9b89c-9689-4c3d-8fa6-18f2ef6dc3ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67951aee09b34949aa5971136e45b895'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:06.386426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1ef5dbca-952f-49de-bea4-6d1ceb7efb90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3967467d9554bc1a717602e7034c79f', 'request_id': 'req-1ef5dbca-952f-49de-bea4-6d1ceb7efb90'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:06.393072 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1ef5dbca-952f-49de-bea4-6d1ceb7efb90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3967467d9554bc1a717602e7034c79f', 'request_id': 'req-1ef5dbca-952f-49de-bea4-6d1ceb7efb90'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:06.400832 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1ef5dbca-952f-49de-bea4-6d1ceb7efb90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3967467d9554bc1a717602e7034c79f', 'request_id': 'req-1ef5dbca-952f-49de-bea4-6d1ceb7efb90'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:06.405481 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-95c9b89c-9689-4c3d-8fa6-18f2ef6dc3ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67951aee09b34949aa5971136e45b895'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:06.504072 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1ef5dbca-952f-49de-bea4-6d1ceb7efb90 admin admin] The token cache is being invalidate because project f3967467d9554bc1a717602e7034c79f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:34:06.507339 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1ef5dbca-952f-49de-bea4-6d1ceb7efb90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:06.507339 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1ef5dbca-952f-49de-bea4-6d1ceb7efb90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:06.507339 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1088/2180] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:34:05 2022] DELETE /identity/v3/projects/f3967467d9554bc1a717602e7034c79f => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:34:06.613705 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-95c9b89c-9689-4c3d-8fa6-18f2ef6dc3ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67951aee09b34949aa5971136e45b895'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:06.614453 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-95c9b89c-9689-4c3d-8fa6-18f2ef6dc3ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67951aee09b34949aa5971136e45b895'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:06.632918 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:06.642298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-95c9b89c-9689-4c3d-8fa6-18f2ef6dc3ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67951aee09b34949aa5971136e45b895'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:06.649780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1093/2181] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:34:05 2022] DELETE /identity/v3/users/67951aee09b34949aa5971136e45b895 => generated 0 bytes in 705 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:07.072444 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5dbca-952f-49de-bea4-6d1ceb7efb90 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:07.073924 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1ef5dbca-952f-49de-bea4-6d1ceb7efb90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:07.132790 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:07.141786 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef5dbca-952f-49de-bea4-6d1ceb7efb90 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:07.204443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dffa39-9a53-4b2e-add7-df7cdd47f452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:07.205111 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dffa39-9a53-4b2e-add7-df7cdd47f452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:07.205385 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dffa39-9a53-4b2e-add7-df7cdd47f452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:07.205820 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dffa39-9a53-4b2e-add7-df7cdd47f452 admin admin] PATH_INFO: `/v3/projects/c1dc0351a5154bfaab9c664adb97dac7` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:07.207413 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dffa39-9a53-4b2e-add7-df7cdd47f452 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1dc0351a5154bfaab9c664adb97dac7)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:07.210366 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dffa39-9a53-4b2e-add7-df7cdd47f452 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:07.345075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-83dffa39-9a53-4b2e-add7-df7cdd47f452 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1dc0351a5154bfaab9c664adb97dac7', 'request_id': 'req-83dffa39-9a53-4b2e-add7-df7cdd47f452'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:07.351225 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-83dffa39-9a53-4b2e-add7-df7cdd47f452 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1dc0351a5154bfaab9c664adb97dac7', 'request_id': 'req-83dffa39-9a53-4b2e-add7-df7cdd47f452'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:07.357317 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-83dffa39-9a53-4b2e-add7-df7cdd47f452 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1dc0351a5154bfaab9c664adb97dac7', 'request_id': 'req-83dffa39-9a53-4b2e-add7-df7cdd47f452'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:07.521657 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-83dffa39-9a53-4b2e-add7-df7cdd47f452 admin admin] The token cache is being invalidate because project c1dc0351a5154bfaab9c664adb97dac7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:34:07.524010 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-83dffa39-9a53-4b2e-add7-df7cdd47f452 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:07.524010 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-83dffa39-9a53-4b2e-add7-df7cdd47f452 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:07.524836 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1089/2182] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:34:07 2022] DELETE /identity/v3/projects/c1dc0351a5154bfaab9c664adb97dac7 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:07.528939 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c9b89c-9689-4c3d-8fa6-18f2ef6dc3ce admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:07.530676 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-95c9b89c-9689-4c3d-8fa6-18f2ef6dc3ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:07.552590 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c9b89c-9689-4c3d-8fa6-18f2ef6dc3ce admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:07.634783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81640e7a-067e-474b-a906-fcf055730cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:07.635487 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81640e7a-067e-474b-a906-fcf055730cbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:07.636403 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81640e7a-067e-474b-a906-fcf055730cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:07.636403 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81640e7a-067e-474b-a906-fcf055730cbd admin admin] PATH_INFO: `/v3/users/7602c5419a85482191cae8015d380209` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:07.637558 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81640e7a-067e-474b-a906-fcf055730cbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7602c5419a85482191cae8015d380209)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:07.639587 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81640e7a-067e-474b-a906-fcf055730cbd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:07.883616 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-81640e7a-067e-474b-a906-fcf055730cbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7602c5419a85482191cae8015d380209'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:07.895595 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-81640e7a-067e-474b-a906-fcf055730cbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7602c5419a85482191cae8015d380209'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:07.995371 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-81640e7a-067e-474b-a906-fcf055730cbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7602c5419a85482191cae8015d380209'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:07.996303 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-81640e7a-067e-474b-a906-fcf055730cbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7602c5419a85482191cae8015d380209'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:08.014458 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-81640e7a-067e-474b-a906-fcf055730cbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7602c5419a85482191cae8015d380209'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:08.022886 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1094/2183] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:34:07 2022] DELETE /identity/v3/users/7602c5419a85482191cae8015d380209 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:08.917756 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dffa39-9a53-4b2e-add7-df7cdd47f452 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:08.919909 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-83dffa39-9a53-4b2e-add7-df7cdd47f452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:08.934548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dffa39-9a53-4b2e-add7-df7cdd47f452 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:08.998376 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4148a8e-afa2-4e71-ba0d-6cdf7f83e76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:08.999039 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4148a8e-afa2-4e71-ba0d-6cdf7f83e76e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:08.999322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4148a8e-afa2-4e71-ba0d-6cdf7f83e76e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:08.999596 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4148a8e-afa2-4e71-ba0d-6cdf7f83e76e admin admin] PATH_INFO: `/v3/projects/ffaa9b3f24fe4791b4d5fcee47255451` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:09.000712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4148a8e-afa2-4e71-ba0d-6cdf7f83e76e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffaa9b3f24fe4791b4d5fcee47255451)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:09.002882 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4148a8e-afa2-4e71-ba0d-6cdf7f83e76e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:09.025315 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:09.149180 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d4148a8e-afa2-4e71-ba0d-6cdf7f83e76e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffaa9b3f24fe4791b4d5fcee47255451', 'request_id': 'req-d4148a8e-afa2-4e71-ba0d-6cdf7f83e76e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:09.154958 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d4148a8e-afa2-4e71-ba0d-6cdf7f83e76e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffaa9b3f24fe4791b4d5fcee47255451', 'request_id': 'req-d4148a8e-afa2-4e71-ba0d-6cdf7f83e76e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:09.168415 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d4148a8e-afa2-4e71-ba0d-6cdf7f83e76e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffaa9b3f24fe4791b4d5fcee47255451', 'request_id': 'req-d4148a8e-afa2-4e71-ba0d-6cdf7f83e76e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:09.248784 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d4148a8e-afa2-4e71-ba0d-6cdf7f83e76e admin admin] The token cache is being invalidate because project ffaa9b3f24fe4791b4d5fcee47255451 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:34:09.249093 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d4148a8e-afa2-4e71-ba0d-6cdf7f83e76e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:09.249745 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d4148a8e-afa2-4e71-ba0d-6cdf7f83e76e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:09.251733 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1090/2184] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:34:08 2022] DELETE /identity/v3/projects/ffaa9b3f24fe4791b4d5fcee47255451 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:09.257244 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f72d6-8946-4131-8e64-3bd6b6d4c252 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:09.257784 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f72d6-8946-4131-8e64-3bd6b6d4c252 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:09.257784 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f72d6-8946-4131-8e64-3bd6b6d4c252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:09.281082 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-849f72d6-8946-4131-8e64-3bd6b6d4c252 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:09.282401 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-849f72d6-8946-4131-8e64-3bd6b6d4c252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:09.343913 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1095/2185] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:09.351988 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391df56-7ecd-4eb3-9b48-2caf77451382 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:09.352279 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391df56-7ecd-4eb3-9b48-2caf77451382 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:09.352555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391df56-7ecd-4eb3-9b48-2caf77451382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:09.373905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e391df56-7ecd-4eb3-9b48-2caf77451382 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:09.375095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e391df56-7ecd-4eb3-9b48-2caf77451382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:09.383924 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1091/2186] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:09.388565 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f72d6-8946-4131-8e64-3bd6b6d4c252 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:09.399003 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f72d6-8946-4131-8e64-3bd6b6d4c252 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:09.403594 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d5abd1-f1f9-4a84-a499-35eafa0588ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:09.404281 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d5abd1-f1f9-4a84-a499-35eafa0588ee admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:09.404561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d5abd1-f1f9-4a84-a499-35eafa0588ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:09.404831 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d5abd1-f1f9-4a84-a499-35eafa0588ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:09.405472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d5abd1-f1f9-4a84-a499-35eafa0588ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:34:09.405769 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d5abd1-f1f9-4a84-a499-35eafa0588ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:09.407383 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d5abd1-f1f9-4a84-a499-35eafa0588ee admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:09.409767 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:09.420744 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1096/2187] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:34:09.425129 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391df56-7ecd-4eb3-9b48-2caf77451382 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:09.434501 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391df56-7ecd-4eb3-9b48-2caf77451382 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:09.439503 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6df4d9-9b4c-4867-af15-c266a52950d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:09.440111 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6df4d9-9b4c-4867-af15-c266a52950d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:09.440393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6df4d9-9b4c-4867-af15-c266a52950d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:09.441973 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6df4d9-9b4c-4867-af15-c266a52950d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:09.441973 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6df4d9-9b4c-4867-af15-c266a52950d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:09.443275 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6df4d9-9b4c-4867-af15-c266a52950d7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:09.538723 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1092/2188] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:34:09 2022] POST /identity/v3/projects => generated 367 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:34:09.539792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d5abd1-f1f9-4a84-a499-35eafa0588ee admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:09.549891 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d5abd1-f1f9-4a84-a499-35eafa0588ee admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:09.610559 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bef392-4a19-4540-a159-326277bd3aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:09.611420 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bef392-4a19-4540-a159-326277bd3aad admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:09.612145 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bef392-4a19-4540-a159-326277bd3aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:09.613818 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bef392-4a19-4540-a159-326277bd3aad admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:09.613818 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bef392-4a19-4540-a159-326277bd3aad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:09.615105 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bef392-4a19-4540-a159-326277bd3aad admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:09.740793 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1097/2189] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:34:09 2022] POST /identity/v3/users => generated 284 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:34:09.744966 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6df4d9-9b4c-4867-af15-c266a52950d7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:09.757810 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6df4d9-9b4c-4867-af15-c266a52950d7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:09.763078 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad9fc9-8128-4eb7-842f-de66c048124b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:09.763825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad9fc9-8128-4eb7-842f-de66c048124b admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:09.764187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad9fc9-8128-4eb7-842f-de66c048124b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:09.764535 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad9fc9-8128-4eb7-842f-de66c048124b admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:09.765266 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad9fc9-8128-4eb7-842f-de66c048124b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:09.766993 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad9fc9-8128-4eb7-842f-de66c048124b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:09.782466 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1093/2190] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:34:09.783869 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bef392-4a19-4540-a159-326277bd3aad admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:09.795166 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bef392-4a19-4540-a159-326277bd3aad admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:09.800476 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0609d05a-dd84-4a78-b119-2a361f81dbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:09.801104 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609d05a-dd84-4a78-b119-2a361f81dbff admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:09.801434 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609d05a-dd84-4a78-b119-2a361f81dbff admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:09.801739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0609d05a-dd84-4a78-b119-2a361f81dbff admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:09.802555 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0609d05a-dd84-4a78-b119-2a361f81dbff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:09.804221 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0609d05a-dd84-4a78-b119-2a361f81dbff admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:09.816707 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1098/2191] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:34:09.821172 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad9fc9-8128-4eb7-842f-de66c048124b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:09.832884 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad9fc9-8128-4eb7-842f-de66c048124b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:09.847623 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c349a-01f0-40c6-9567-8c51bcfa9fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:09.848616 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c349a-01f0-40c6-9567-8c51bcfa9fd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:09.849066 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c349a-01f0-40c6-9567-8c51bcfa9fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:09.849440 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c349a-01f0-40c6-9567-8c51bcfa9fd0 admin admin] PATH_INFO: `/v3/projects/017244586132424d96a0dcb3d7d0828a/users/41fb6ec548094f6eb60714d4da802146/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:09.866211 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c349a-01f0-40c6-9567-8c51bcfa9fd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=017244586132424d96a0dcb3d7d0828a, user_id=41fb6ec548094f6eb60714d4da802146, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:09.869441 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c349a-01f0-40c6-9567-8c51bcfa9fd0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:09.999267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1094/2192] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:34:09 2022] PUT /identity/v3/projects/017244586132424d96a0dcb3d7d0828a/users/41fb6ec548094f6eb60714d4da802146/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:34:10.011228 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fdb579-78bd-43c8-ad68-cefe61c4f48e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:10.011537 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fdb579-78bd-43c8-ad68-cefe61c4f48e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:10.011872 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fdb579-78bd-43c8-ad68-cefe61c4f48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:10.043912 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f6fdb579-78bd-43c8-ad68-cefe61c4f48e None None] MFA Rules not processed for user `41fb6ec548094f6eb60714d4da802146`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:10.045128 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f6fdb579-78bd-43c8-ad68-cefe61c4f48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:10.124757 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1099/2193] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:10.136609 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c349a-01f0-40c6-9567-8c51bcfa9fd0 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:10.138583 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1e6c349a-01f0-40c6-9567-8c51bcfa9fd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:10.204153 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c349a-01f0-40c6-9567-8c51bcfa9fd0 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:10.210774 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d0da5-30c9-4196-af20-ebb29223cd54 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b555bb94ce44ce918cbf2a012576a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:10.211041 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d0da5-30c9-4196-af20-ebb29223cd54 service glance] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:10.211264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d0da5-30c9-4196-af20-ebb29223cd54 service glance] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:10.211540 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d0da5-30c9-4196-af20-ebb29223cd54 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:10.223823 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d0da5-30c9-4196-af20-ebb29223cd54 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:10.226029 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d0da5-30c9-4196-af20-ebb29223cd54 service glance] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:10.239037 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1095/2194] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:34:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:34:12.197115 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fdb579-78bd-43c8-ad68-cefe61c4f48e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:12.198613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f6fdb579-78bd-43c8-ad68-cefe61c4f48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:12.266427 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fdb579-78bd-43c8-ad68-cefe61c4f48e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:12.271075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8453d9-5abf-40c6-b395-17c5d97a3aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:12.271918 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8453d9-5abf-40c6-b395-17c5d97a3aa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:12.272284 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8453d9-5abf-40c6-b395-17c5d97a3aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:12.272583 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8453d9-5abf-40c6-b395-17c5d97a3aa6 admin admin] PATH_INFO: `/v3/users/d4d4f092761d44a2a30f0e1d044888b6` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:12.274035 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8453d9-5abf-40c6-b395-17c5d97a3aa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4d4f092761d44a2a30f0e1d044888b6)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:12.276035 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8453d9-5abf-40c6-b395-17c5d97a3aa6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:12.280980 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:12.518183 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ea8453d9-5abf-40c6-b395-17c5d97a3aa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4d4f092761d44a2a30f0e1d044888b6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:12.526262 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ea8453d9-5abf-40c6-b395-17c5d97a3aa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4d4f092761d44a2a30f0e1d044888b6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:12.584872 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ea8453d9-5abf-40c6-b395-17c5d97a3aa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4d4f092761d44a2a30f0e1d044888b6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:12.585484 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ea8453d9-5abf-40c6-b395-17c5d97a3aa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4d4f092761d44a2a30f0e1d044888b6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:12.598377 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ea8453d9-5abf-40c6-b395-17c5d97a3aa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4d4f092761d44a2a30f0e1d044888b6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:12.605141 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1100/2195] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:34:12 2022] DELETE /identity/v3/users/d4d4f092761d44a2a30f0e1d044888b6 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:12.846553 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d0da5-30c9-4196-af20-ebb29223cd54 service glance] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:12.857890 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d0da5-30c9-4196-af20-ebb29223cd54 service glance] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:12.910137 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:12.922100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc26f0-f13f-44fc-a4f9-18bfaf0b92bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:12.922814 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc26f0-f13f-44fc-a4f9-18bfaf0b92bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:12.923093 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc26f0-f13f-44fc-a4f9-18bfaf0b92bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:12.923363 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc26f0-f13f-44fc-a4f9-18bfaf0b92bb admin admin] PATH_INFO: `/v3/projects/e553fe4257cc4983ac82027b05a53e65` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:12.924504 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc26f0-f13f-44fc-a4f9-18bfaf0b92bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e553fe4257cc4983ac82027b05a53e65)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:12.926379 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dc26f0-f13f-44fc-a4f9-18bfaf0b92bb admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:13.036338 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-03dc26f0-f13f-44fc-a4f9-18bfaf0b92bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e553fe4257cc4983ac82027b05a53e65', 'request_id': 'req-03dc26f0-f13f-44fc-a4f9-18bfaf0b92bb'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:13.041655 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-03dc26f0-f13f-44fc-a4f9-18bfaf0b92bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e553fe4257cc4983ac82027b05a53e65', 'request_id': 'req-03dc26f0-f13f-44fc-a4f9-18bfaf0b92bb'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:13.050667 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-03dc26f0-f13f-44fc-a4f9-18bfaf0b92bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e553fe4257cc4983ac82027b05a53e65', 'request_id': 'req-03dc26f0-f13f-44fc-a4f9-18bfaf0b92bb'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:13.133785 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-03dc26f0-f13f-44fc-a4f9-18bfaf0b92bb admin admin] The token cache is being invalidate because project e553fe4257cc4983ac82027b05a53e65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:34:13.134083 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-03dc26f0-f13f-44fc-a4f9-18bfaf0b92bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:13.134763 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-03dc26f0-f13f-44fc-a4f9-18bfaf0b92bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:13.136349 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1096/2196] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:34:12 2022] DELETE /identity/v3/projects/e553fe4257cc4983ac82027b05a53e65 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:13.141318 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7135cd11-879f-445c-b4bd-fa57588c6615 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:13.141661 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7135cd11-879f-445c-b4bd-fa57588c6615 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:13.141943 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7135cd11-879f-445c-b4bd-fa57588c6615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:13.162920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-7135cd11-879f-445c-b4bd-fa57588c6615 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:13.164137 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7135cd11-879f-445c-b4bd-fa57588c6615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:13.223807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1101/2197] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:13.231672 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e87f81-cb34-40a0-81c8-04f5083fe864 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:13.231962 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e87f81-cb34-40a0-81c8-04f5083fe864 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:13.232240 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e87f81-cb34-40a0-81c8-04f5083fe864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:13.252766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a9e87f81-cb34-40a0-81c8-04f5083fe864 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:13.253996 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a9e87f81-cb34-40a0-81c8-04f5083fe864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:13.262524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1097/2198] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:13.266651 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7135cd11-879f-445c-b4bd-fa57588c6615 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:13.276287 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7135cd11-879f-445c-b4bd-fa57588c6615 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:13.280856 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ea128-9584-4a7f-86bf-9382897f0320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:13.281450 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ea128-9584-4a7f-86bf-9382897f0320 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:13.281729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ea128-9584-4a7f-86bf-9382897f0320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:13.282000 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ea128-9584-4a7f-86bf-9382897f0320 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:13.282641 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ea128-9584-4a7f-86bf-9382897f0320 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:34:13.282977 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ea128-9584-4a7f-86bf-9382897f0320 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:13.286143 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ea128-9584-4a7f-86bf-9382897f0320 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:13.296314 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1102/2199] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:34:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:34:13.300487 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e87f81-cb34-40a0-81c8-04f5083fe864 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:13.314840 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e87f81-cb34-40a0-81c8-04f5083fe864 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:13.314968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc4a076-52d5-491c-8cc6-baed561def93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:13.315559 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4a076-52d5-491c-8cc6-baed561def93 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:13.315841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4a076-52d5-491c-8cc6-baed561def93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:13.316116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4a076-52d5-491c-8cc6-baed561def93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:13.316762 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc4a076-52d5-491c-8cc6-baed561def93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:13.319228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc4a076-52d5-491c-8cc6-baed561def93 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:13.403092 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1098/2200] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:34:13 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:34:13.407094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ea128-9584-4a7f-86bf-9382897f0320 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:13.417010 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ea128-9584-4a7f-86bf-9382897f0320 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:13.475022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56efd50-902b-4705-a57a-b658c785657c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:13.475625 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56efd50-902b-4705-a57a-b658c785657c admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:13.475899 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56efd50-902b-4705-a57a-b658c785657c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:13.476165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56efd50-902b-4705-a57a-b658c785657c admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:13.476794 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56efd50-902b-4705-a57a-b658c785657c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:13.478667 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56efd50-902b-4705-a57a-b658c785657c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:13.588812 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:13.588812 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1103/2201] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:34:13 2022] POST /identity/v3/users => generated 290 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:34:13.589711 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc4a076-52d5-491c-8cc6-baed561def93 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:13.601372 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc4a076-52d5-491c-8cc6-baed561def93 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:13.606034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94280d86-0fe4-43a7-a318-f9bd31bf4587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:13.611849 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94280d86-0fe4-43a7-a318-f9bd31bf4587 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:13.611849 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94280d86-0fe4-43a7-a318-f9bd31bf4587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:13.611849 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94280d86-0fe4-43a7-a318-f9bd31bf4587 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:13.611849 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94280d86-0fe4-43a7-a318-f9bd31bf4587 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:13.611849 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94280d86-0fe4-43a7-a318-f9bd31bf4587 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:13.620286 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1099/2202] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:34:13.624499 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56efd50-902b-4705-a57a-b658c785657c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:13.634727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56efd50-902b-4705-a57a-b658c785657c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:13.639422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb52f7-d6fb-4525-801c-c7d4d1356568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:13.640050 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb52f7-d6fb-4525-801c-c7d4d1356568 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:13.640334 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb52f7-d6fb-4525-801c-c7d4d1356568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:13.640607 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb52f7-d6fb-4525-801c-c7d4d1356568 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:13.641281 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fb52f7-d6fb-4525-801c-c7d4d1356568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:13.642906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fb52f7-d6fb-4525-801c-c7d4d1356568 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:13.654276 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1104/2203] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:34:13.658239 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94280d86-0fe4-43a7-a318-f9bd31bf4587 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:13.668905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94280d86-0fe4-43a7-a318-f9bd31bf4587 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:13.673598 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad17be-87bf-40e2-aa47-4e2e8f6c4527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:13.674259 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad17be-87bf-40e2-aa47-4e2e8f6c4527 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:13.674541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad17be-87bf-40e2-aa47-4e2e8f6c4527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:13.674850 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ad17be-87bf-40e2-aa47-4e2e8f6c4527 admin admin] PATH_INFO: `/v3/projects/5795af30e2d04e1f87ba8a5dda346824/users/300cfe69b06a4d83a8aad0eb73df9408/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:13.691895 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad17be-87bf-40e2-aa47-4e2e8f6c4527 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5795af30e2d04e1f87ba8a5dda346824, user_id=300cfe69b06a4d83a8aad0eb73df9408, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:13.694597 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ad17be-87bf-40e2-aa47-4e2e8f6c4527 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:13.796026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1100/2204] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:34:13 2022] PUT /identity/v3/projects/5795af30e2d04e1f87ba8a5dda346824/users/300cfe69b06a4d83a8aad0eb73df9408/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:34:13.806641 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821bb4da-cf3f-4cec-bb98-3e184d43c4ce None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:13.806959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821bb4da-cf3f-4cec-bb98-3e184d43c4ce None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:13.807233 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821bb4da-cf3f-4cec-bb98-3e184d43c4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:13.834451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-821bb4da-cf3f-4cec-bb98-3e184d43c4ce None None] MFA Rules not processed for user `300cfe69b06a4d83a8aad0eb73df9408`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:13.835718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-821bb4da-cf3f-4cec-bb98-3e184d43c4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:13.906064 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1105/2205] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:13.910456 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad17be-87bf-40e2-aa47-4e2e8f6c4527 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:13.972906 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ad17be-87bf-40e2-aa47-4e2e8f6c4527 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:13.978071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c741c1-aba5-4db9-959e-43ac1f626f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:13.978671 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c741c1-aba5-4db9-959e-43ac1f626f2b admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:13.978980 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c741c1-aba5-4db9-959e-43ac1f626f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:13.979250 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c741c1-aba5-4db9-959e-43ac1f626f2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:13.979873 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c741c1-aba5-4db9-959e-43ac1f626f2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:13.981756 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c741c1-aba5-4db9-959e-43ac1f626f2b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:14.100717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:14.106727 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1101/2206] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:34:13 2022] POST /identity/v3/projects => generated 379 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:34:14.110676 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821bb4da-cf3f-4cec-bb98-3e184d43c4ce None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:14.121354 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821bb4da-cf3f-4cec-bb98-3e184d43c4ce None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:14.178369 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00663707-5c5f-4281-8bba-999b3b8f7ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:14.179029 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00663707-5c5f-4281-8bba-999b3b8f7ff9 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:14.179351 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00663707-5c5f-4281-8bba-999b3b8f7ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:14.179621 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00663707-5c5f-4281-8bba-999b3b8f7ff9 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:14.180243 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00663707-5c5f-4281-8bba-999b3b8f7ff9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:14.182140 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00663707-5c5f-4281-8bba-999b3b8f7ff9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:14.298470 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1106/2207] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:34:14 2022] POST /identity/v3/users => generated 296 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:34:14.302436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c741c1-aba5-4db9-959e-43ac1f626f2b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:14.312626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c741c1-aba5-4db9-959e-43ac1f626f2b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:14.317173 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152fcb11-1115-44a2-9526-4a8ea227653f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:14.317766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fcb11-1115-44a2-9526-4a8ea227653f admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:14.318048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fcb11-1115-44a2-9526-4a8ea227653f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:14.318852 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fcb11-1115-44a2-9526-4a8ea227653f admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:14.319492 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152fcb11-1115-44a2-9526-4a8ea227653f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:14.321070 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152fcb11-1115-44a2-9526-4a8ea227653f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:14.331393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1102/2208] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:34:14.335938 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00663707-5c5f-4281-8bba-999b3b8f7ff9 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:14.345941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00663707-5c5f-4281-8bba-999b3b8f7ff9 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:14.350586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb5a08-3a42-4aef-a4d8-aab2fdf71dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:14.351304 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb5a08-3a42-4aef-a4d8-aab2fdf71dd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:14.351529 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb5a08-3a42-4aef-a4d8-aab2fdf71dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:14.351803 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb5a08-3a42-4aef-a4d8-aab2fdf71dd6 admin admin] PATH_INFO: `/v3/projects/cbdce41fbf784aa2951c9c6362106c9f/users/a54b51e13cfc4a0e965fd52c9ff9b82c/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:14.366894 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccb5a08-3a42-4aef-a4d8-aab2fdf71dd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbdce41fbf784aa2951c9c6362106c9f, user_id=a54b51e13cfc4a0e965fd52c9ff9b82c, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:14.369518 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccb5a08-3a42-4aef-a4d8-aab2fdf71dd6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:14.436561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1107/2209] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:34:14 2022] PUT /identity/v3/projects/cbdce41fbf784aa2951c9c6362106c9f/users/a54b51e13cfc4a0e965fd52c9ff9b82c/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:14.440612 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152fcb11-1115-44a2-9526-4a8ea227653f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:14.451133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152fcb11-1115-44a2-9526-4a8ea227653f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:14.516769 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6968a55-531a-4d90-b6af-12bcdb7154bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:14.517482 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6968a55-531a-4d90-b6af-12bcdb7154bf admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:14.517662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6968a55-531a-4d90-b6af-12bcdb7154bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:14.517936 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6968a55-531a-4d90-b6af-12bcdb7154bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:14.530255 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6968a55-531a-4d90-b6af-12bcdb7154bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:14.530255 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6968a55-531a-4d90-b6af-12bcdb7154bf admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:14.530627 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1103/2210] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:34:14.534690 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb5a08-3a42-4aef-a4d8-aab2fdf71dd6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:14.544997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb5a08-3a42-4aef-a4d8-aab2fdf71dd6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:14.549504 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb034f81-f44b-4b2a-9fc0-7153a75b125c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:14.550142 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb034f81-f44b-4b2a-9fc0-7153a75b125c admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:14.550418 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb034f81-f44b-4b2a-9fc0-7153a75b125c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:14.550686 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb034f81-f44b-4b2a-9fc0-7153a75b125c admin admin] PATH_INFO: `/v3/projects/cbdce41fbf784aa2951c9c6362106c9f/users/a54b51e13cfc4a0e965fd52c9ff9b82c/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:14.552578 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb034f81-f44b-4b2a-9fc0-7153a75b125c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbdce41fbf784aa2951c9c6362106c9f, user_id=a54b51e13cfc4a0e965fd52c9ff9b82c, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:14.555230 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb034f81-f44b-4b2a-9fc0-7153a75b125c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:14.565145 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1108/2211] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:34:14 2022] PUT /identity/v3/projects/cbdce41fbf784aa2951c9c6362106c9f/users/a54b51e13cfc4a0e965fd52c9ff9b82c/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:14.576724 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520cd636-2fc0-42d6-b632-e983cd341e3a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:14.577015 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520cd636-2fc0-42d6-b632-e983cd341e3a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:14.577296 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520cd636-2fc0-42d6-b632-e983cd341e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:14.604937 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-520cd636-2fc0-42d6-b632-e983cd341e3a None None] MFA Rules not processed for user `a54b51e13cfc4a0e965fd52c9ff9b82c`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:14.606083 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-520cd636-2fc0-42d6-b632-e983cd341e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:14.673832 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1104/2212] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:14 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:14.685625 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb034f81-f44b-4b2a-9fc0-7153a75b125c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:14.687301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bb034f81-f44b-4b2a-9fc0-7153a75b125c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:14.722201 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:14.745970 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb034f81-f44b-4b2a-9fc0-7153a75b125c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:14.749689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7597de3-e533-4f43-afa0-aca1b918af27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:14.750295 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7597de3-e533-4f43-afa0-aca1b918af27 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:14.750571 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7597de3-e533-4f43-afa0-aca1b918af27 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:14.751141 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7597de3-e533-4f43-afa0-aca1b918af27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:14.761581 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7597de3-e533-4f43-afa0-aca1b918af27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:14.763068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7597de3-e533-4f43-afa0-aca1b918af27 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:14.773441 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1109/2213] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:34:14 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:34:14.948190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520cd636-2fc0-42d6-b632-e983cd341e3a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:14.958051 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520cd636-2fc0-42d6-b632-e983cd341e3a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:14.961754 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd15d886-70c8-42c1-af10-47b4f35a182e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:14.962350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15d886-70c8-42c1-af10-47b4f35a182e service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:14.962627 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15d886-70c8-42c1-af10-47b4f35a182e service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:14.962928 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd15d886-70c8-42c1-af10-47b4f35a182e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:15.020553 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd15d886-70c8-42c1-af10-47b4f35a182e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:15.022058 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd15d886-70c8-42c1-af10-47b4f35a182e service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:15.030921 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1105/2214] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:34:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:34:23.904322 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7597de3-e533-4f43-afa0-aca1b918af27 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:23.905948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d7597de3-e533-4f43-afa0-aca1b918af27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:23.997215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7597de3-e533-4f43-afa0-aca1b918af27 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:24.002493 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d390d0d-50f7-4ba6-aefc-3ff26197ceff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:24.003047 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d390d0d-50f7-4ba6-aefc-3ff26197ceff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:24.003325 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d390d0d-50f7-4ba6-aefc-3ff26197ceff admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:24.012173 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d390d0d-50f7-4ba6-aefc-3ff26197ceff admin admin] PATH_INFO: `/v3/users/0603f354493a4f949a2bdf701ca1465b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:24.012173 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d390d0d-50f7-4ba6-aefc-3ff26197ceff admin admin] RBAC: Authorizing `identity:delete_user(user_id=0603f354493a4f949a2bdf701ca1465b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:24.012173 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d390d0d-50f7-4ba6-aefc-3ff26197ceff admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:24.202859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1d390d0d-50f7-4ba6-aefc-3ff26197ceff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0603f354493a4f949a2bdf701ca1465b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:24.210110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1d390d0d-50f7-4ba6-aefc-3ff26197ceff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0603f354493a4f949a2bdf701ca1465b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:24.332625 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1d390d0d-50f7-4ba6-aefc-3ff26197ceff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0603f354493a4f949a2bdf701ca1465b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:24.333174 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1d390d0d-50f7-4ba6-aefc-3ff26197ceff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0603f354493a4f949a2bdf701ca1465b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:24.346391 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1d390d0d-50f7-4ba6-aefc-3ff26197ceff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0603f354493a4f949a2bdf701ca1465b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:24.352929 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1110/2215] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:34:23 2022] DELETE /identity/v3/users/0603f354493a4f949a2bdf701ca1465b => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:24.357349 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10f479-ed31-40fc-9e8a-040355f9ca3c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:24.358912 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10f479-ed31-40fc-9e8a-040355f9ca3c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:24.358912 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10f479-ed31-40fc-9e8a-040355f9ca3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:24.377070 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ab10f479-ed31-40fc-9e8a-040355f9ca3c None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:24.378279 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ab10f479-ed31-40fc-9e8a-040355f9ca3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:24.382343 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:24.436655 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1106/2216] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:24.446355 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d390d0d-50f7-4ba6-aefc-3ff26197ceff admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:24.447794 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1d390d0d-50f7-4ba6-aefc-3ff26197ceff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:24.503907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d390d0d-50f7-4ba6-aefc-3ff26197ceff admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:24.507482 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc221fc4-b919-402f-8646-45c5fe46d1b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:24.508075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc221fc4-b919-402f-8646-45c5fe46d1b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:24.508354 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc221fc4-b919-402f-8646-45c5fe46d1b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:24.508624 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc221fc4-b919-402f-8646-45c5fe46d1b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:24.515399 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:24.517623 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc221fc4-b919-402f-8646-45c5fe46d1b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:24.519069 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc221fc4-b919-402f-8646-45c5fe46d1b2 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:24.528784 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1111/2217] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:34:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:34:24.922470 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10f479-ed31-40fc-9e8a-040355f9ca3c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:24.924164 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ab10f479-ed31-40fc-9e8a-040355f9ca3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:24.937430 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10f479-ed31-40fc-9e8a-040355f9ca3c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:24.942217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc47981-d409-4ddf-b9e2-45f1ae69b5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:24.943198 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc47981-d409-4ddf-b9e2-45f1ae69b5b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:24.943475 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc47981-d409-4ddf-b9e2-45f1ae69b5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:24.946007 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc47981-d409-4ddf-b9e2-45f1ae69b5b5 admin admin] PATH_INFO: `/v3/projects/9ccced06207840208e83ada14514c7d0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:24.946192 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc47981-d409-4ddf-b9e2-45f1ae69b5b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ccced06207840208e83ada14514c7d0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:24.948184 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc47981-d409-4ddf-b9e2-45f1ae69b5b5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:25.088098 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6dc47981-d409-4ddf-b9e2-45f1ae69b5b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ccced06207840208e83ada14514c7d0', 'request_id': 'req-6dc47981-d409-4ddf-b9e2-45f1ae69b5b5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:25.093504 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6dc47981-d409-4ddf-b9e2-45f1ae69b5b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ccced06207840208e83ada14514c7d0', 'request_id': 'req-6dc47981-d409-4ddf-b9e2-45f1ae69b5b5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:25.099627 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6dc47981-d409-4ddf-b9e2-45f1ae69b5b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ccced06207840208e83ada14514c7d0', 'request_id': 'req-6dc47981-d409-4ddf-b9e2-45f1ae69b5b5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:25.190590 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6dc47981-d409-4ddf-b9e2-45f1ae69b5b5 admin admin] The token cache is being invalidate because project 9ccced06207840208e83ada14514c7d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:34:25.192558 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6dc47981-d409-4ddf-b9e2-45f1ae69b5b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:25.193397 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6dc47981-d409-4ddf-b9e2-45f1ae69b5b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:25.195352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1107/2218] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:34:24 2022] DELETE /identity/v3/projects/9ccced06207840208e83ada14514c7d0 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:25.204143 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc221fc4-b919-402f-8646-45c5fe46d1b2 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:25.216626 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc221fc4-b919-402f-8646-45c5fe46d1b2 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:25.276405 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c0f8df-5faa-490d-86ff-3f5e8c109e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:25.277033 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0f8df-5faa-490d-86ff-3f5e8c109e84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:25.277315 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0f8df-5faa-490d-86ff-3f5e8c109e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:25.277589 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c0f8df-5faa-490d-86ff-3f5e8c109e84 admin admin] PATH_INFO: `/v3/users/04bcc80e89cb412d953b5dda5fdc04e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:25.278732 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c0f8df-5faa-490d-86ff-3f5e8c109e84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04bcc80e89cb412d953b5dda5fdc04e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:25.280748 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c0f8df-5faa-490d-86ff-3f5e8c109e84 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:25.528855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-19c0f8df-5faa-490d-86ff-3f5e8c109e84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04bcc80e89cb412d953b5dda5fdc04e1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:25.535640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-19c0f8df-5faa-490d-86ff-3f5e8c109e84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04bcc80e89cb412d953b5dda5fdc04e1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:25.600547 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-19c0f8df-5faa-490d-86ff-3f5e8c109e84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04bcc80e89cb412d953b5dda5fdc04e1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:25.601174 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-19c0f8df-5faa-490d-86ff-3f5e8c109e84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04bcc80e89cb412d953b5dda5fdc04e1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:25.613357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-19c0f8df-5faa-490d-86ff-3f5e8c109e84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04bcc80e89cb412d953b5dda5fdc04e1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:25.619648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1112/2219] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:34:25 2022] DELETE /identity/v3/users/04bcc80e89cb412d953b5dda5fdc04e1 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:26.013259 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc47981-d409-4ddf-b9e2-45f1ae69b5b5 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:26.014696 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6dc47981-d409-4ddf-b9e2-45f1ae69b5b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:26.029030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc47981-d409-4ddf-b9e2-45f1ae69b5b5 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:26.098831 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e72e7-d176-43dc-8435-68bb8ed6a1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:26.099496 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e72e7-d176-43dc-8435-68bb8ed6a1b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:26.099761 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e72e7-d176-43dc-8435-68bb8ed6a1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:26.100041 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e72e7-d176-43dc-8435-68bb8ed6a1b1 admin admin] PATH_INFO: `/v3/projects/6767f0a3ee32455ab100ce4b9272358a` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:26.101240 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3e72e7-d176-43dc-8435-68bb8ed6a1b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6767f0a3ee32455ab100ce4b9272358a)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:26.103193 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3e72e7-d176-43dc-8435-68bb8ed6a1b1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:26.226048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:26.232114 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2b3e72e7-d176-43dc-8435-68bb8ed6a1b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6767f0a3ee32455ab100ce4b9272358a', 'request_id': 'req-2b3e72e7-d176-43dc-8435-68bb8ed6a1b1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:26.237578 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2b3e72e7-d176-43dc-8435-68bb8ed6a1b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6767f0a3ee32455ab100ce4b9272358a', 'request_id': 'req-2b3e72e7-d176-43dc-8435-68bb8ed6a1b1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:26.244715 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2b3e72e7-d176-43dc-8435-68bb8ed6a1b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6767f0a3ee32455ab100ce4b9272358a', 'request_id': 'req-2b3e72e7-d176-43dc-8435-68bb8ed6a1b1'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:26.315348 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2b3e72e7-d176-43dc-8435-68bb8ed6a1b1 admin admin] The token cache is being invalidate because project 6767f0a3ee32455ab100ce4b9272358a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:34:26.315564 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2b3e72e7-d176-43dc-8435-68bb8ed6a1b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:26.316184 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2b3e72e7-d176-43dc-8435-68bb8ed6a1b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:26.317900 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1108/2220] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:34:26 2022] DELETE /identity/v3/projects/6767f0a3ee32455ab100ce4b9272358a => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:26.323021 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e938e3-8626-4f24-8a4c-cb4b43d43ca8 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:26.323501 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e938e3-8626-4f24-8a4c-cb4b43d43ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:26.323590 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e938e3-8626-4f24-8a4c-cb4b43d43ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:26.343041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b1e938e3-8626-4f24-8a4c-cb4b43d43ca8 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:26.344239 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b1e938e3-8626-4f24-8a4c-cb4b43d43ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:26.402909 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1113/2221] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:34:26.410866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcc6e0-c79a-4d03-baa4-17d0857558e6 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:26.411157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcc6e0-c79a-4d03-baa4-17d0857558e6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:26.411437 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcc6e0-c79a-4d03-baa4-17d0857558e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:26.433444 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-57dcc6e0-c79a-4d03-baa4-17d0857558e6 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:26.434747 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-57dcc6e0-c79a-4d03-baa4-17d0857558e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:26.442734 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1109/2222] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:26.451978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e938e3-8626-4f24-8a4c-cb4b43d43ca8 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:26.461463 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e938e3-8626-4f24-8a4c-cb4b43d43ca8 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:26.465954 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2153310-66fb-4ca8-b908-89abefb9ca8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:26.466548 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2153310-66fb-4ca8-b908-89abefb9ca8c admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:26.466876 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2153310-66fb-4ca8-b908-89abefb9ca8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:26.467149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2153310-66fb-4ca8-b908-89abefb9ca8c admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:26.467792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2153310-66fb-4ca8-b908-89abefb9ca8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:34:26.468093 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2153310-66fb-4ca8-b908-89abefb9ca8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:26.469653 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2153310-66fb-4ca8-b908-89abefb9ca8c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:26.481413 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1114/2223] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:34:26.485532 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcc6e0-c79a-4d03-baa4-17d0857558e6 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:26.495322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcc6e0-c79a-4d03-baa4-17d0857558e6 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:26.499841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342cce1-92a7-4f94-a515-f9097d3c615c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:26.500681 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342cce1-92a7-4f94-a515-f9097d3c615c admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:26.500955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342cce1-92a7-4f94-a515-f9097d3c615c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:26.501221 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342cce1-92a7-4f94-a515-f9097d3c615c admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:26.501862 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c342cce1-92a7-4f94-a515-f9097d3c615c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:26.503768 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c342cce1-92a7-4f94-a515-f9097d3c615c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:26.596457 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1110/2224] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:34:26 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:34:26.601324 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2153310-66fb-4ca8-b908-89abefb9ca8c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:26.606387 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:26.611943 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2153310-66fb-4ca8-b908-89abefb9ca8c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:26.668880 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e34ca-514d-4238-a291-88e31af8bd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:26.669483 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e34ca-514d-4238-a291-88e31af8bd17 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:26.669764 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e34ca-514d-4238-a291-88e31af8bd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:26.670037 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e34ca-514d-4238-a291-88e31af8bd17 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:26.670660 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e34ca-514d-4238-a291-88e31af8bd17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:26.672587 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e34ca-514d-4238-a291-88e31af8bd17 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:26.748968 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1115/2225] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:34:26 2022] POST /identity/v3/users => generated 283 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:34:26.753730 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342cce1-92a7-4f94-a515-f9097d3c615c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:26.764048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342cce1-92a7-4f94-a515-f9097d3c615c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:26.768654 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb90eb6-e753-4927-a724-55e0cc01ac6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:26.769251 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb90eb6-e753-4927-a724-55e0cc01ac6d admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:26.769542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb90eb6-e753-4927-a724-55e0cc01ac6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:26.769807 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb90eb6-e753-4927-a724-55e0cc01ac6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:26.770448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb90eb6-e753-4927-a724-55e0cc01ac6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:26.772140 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb90eb6-e753-4927-a724-55e0cc01ac6d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:26.782873 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1111/2226] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:34:26.786539 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e34ca-514d-4238-a291-88e31af8bd17 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:26.796972 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e34ca-514d-4238-a291-88e31af8bd17 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:26.801477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d26a86-12ae-45d0-8b8e-1e2f8550aafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:26.802058 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d26a86-12ae-45d0-8b8e-1e2f8550aafc admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:26.802338 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d26a86-12ae-45d0-8b8e-1e2f8550aafc admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:26.802605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d26a86-12ae-45d0-8b8e-1e2f8550aafc admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:26.803275 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d26a86-12ae-45d0-8b8e-1e2f8550aafc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:26.804907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d26a86-12ae-45d0-8b8e-1e2f8550aafc admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:26.815286 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1116/2227] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:34:26.819249 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb90eb6-e753-4927-a724-55e0cc01ac6d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:26.829095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb90eb6-e753-4927-a724-55e0cc01ac6d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:26.833879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ebbff-9eb3-4f1c-aadb-ce48d1d890ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:26.834526 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ebbff-9eb3-4f1c-aadb-ce48d1d890ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:26.835366 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ebbff-9eb3-4f1c-aadb-ce48d1d890ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:26.835659 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ebbff-9eb3-4f1c-aadb-ce48d1d890ed admin admin] PATH_INFO: `/v3/projects/d7ec12d875ef449eb2b95bd275c4c6b0/users/186a492ddee442b495cd8fce575f570c/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:26.851117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ebbff-9eb3-4f1c-aadb-ce48d1d890ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7ec12d875ef449eb2b95bd275c4c6b0, user_id=186a492ddee442b495cd8fce575f570c, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:26.853762 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ebbff-9eb3-4f1c-aadb-ce48d1d890ed admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:26.932926 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1112/2228] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:34:26 2022] PUT /identity/v3/projects/d7ec12d875ef449eb2b95bd275c4c6b0/users/186a492ddee442b495cd8fce575f570c/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:26.943067 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf74281-e952-4c67-8b83-5bd5b2be93ef None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:26.943355 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf74281-e952-4c67-8b83-5bd5b2be93ef None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:26.943632 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf74281-e952-4c67-8b83-5bd5b2be93ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:26.970909 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6bf74281-e952-4c67-8b83-5bd5b2be93ef None None] MFA Rules not processed for user `186a492ddee442b495cd8fce575f570c`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:26.972119 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6bf74281-e952-4c67-8b83-5bd5b2be93ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:27.033941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1117/2229] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:27.038965 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ebbff-9eb3-4f1c-aadb-ce48d1d890ed admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:27.104614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ebbff-9eb3-4f1c-aadb-ce48d1d890ed admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:27.109257 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874930e9-5fbe-45ae-96cb-12aee4e1dc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:27.109856 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874930e9-5fbe-45ae-96cb-12aee4e1dc18 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:27.110132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874930e9-5fbe-45ae-96cb-12aee4e1dc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:27.110408 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874930e9-5fbe-45ae-96cb-12aee4e1dc18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:27.111064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874930e9-5fbe-45ae-96cb-12aee4e1dc18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:27.112942 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874930e9-5fbe-45ae-96cb-12aee4e1dc18 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:27.219716 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1113/2230] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:34:27 2022] POST /identity/v3/projects => generated 365 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:34:27.223615 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf74281-e952-4c67-8b83-5bd5b2be93ef None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:27.234941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf74281-e952-4c67-8b83-5bd5b2be93ef None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:27.293743 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc95b4-b1cc-44a6-9d9a-ea9e4317117b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:27.294370 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc95b4-b1cc-44a6-9d9a-ea9e4317117b admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:27.294656 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc95b4-b1cc-44a6-9d9a-ea9e4317117b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:27.294980 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc95b4-b1cc-44a6-9d9a-ea9e4317117b admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:27.295648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc95b4-b1cc-44a6-9d9a-ea9e4317117b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:27.297584 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc95b4-b1cc-44a6-9d9a-ea9e4317117b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:27.414241 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:27.419367 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1118/2231] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:34:27 2022] POST /identity/v3/users => generated 289 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:34:27.423394 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874930e9-5fbe-45ae-96cb-12aee4e1dc18 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:27.433848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874930e9-5fbe-45ae-96cb-12aee4e1dc18 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:27.438393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a7cfd-1734-4b0b-96b3-a3fe640f7892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:27.439033 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a7cfd-1734-4b0b-96b3-a3fe640f7892 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:27.439310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a7cfd-1734-4b0b-96b3-a3fe640f7892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:27.439578 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a7cfd-1734-4b0b-96b3-a3fe640f7892 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:27.440291 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a7cfd-1734-4b0b-96b3-a3fe640f7892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:27.441921 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a7cfd-1734-4b0b-96b3-a3fe640f7892 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:27.452916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1114/2232] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:34:27.457907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc95b4-b1cc-44a6-9d9a-ea9e4317117b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:27.468318 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc95b4-b1cc-44a6-9d9a-ea9e4317117b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:27.473067 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a0c5e-714a-4ff3-acff-9ce7610a034d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:27.473961 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a0c5e-714a-4ff3-acff-9ce7610a034d admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:27.474668 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a0c5e-714a-4ff3-acff-9ce7610a034d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:27.474982 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a0c5e-714a-4ff3-acff-9ce7610a034d admin admin] PATH_INFO: `/v3/projects/b964110459f54020ae3963e8aba796f4/users/3ca05db84f454d46a741e4414ef3586d/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:27.490084 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a0c5e-714a-4ff3-acff-9ce7610a034d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b964110459f54020ae3963e8aba796f4, user_id=3ca05db84f454d46a741e4414ef3586d, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:27.492842 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a0c5e-714a-4ff3-acff-9ce7610a034d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:27.564912 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1119/2233] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:34:27 2022] PUT /identity/v3/projects/b964110459f54020ae3963e8aba796f4/users/3ca05db84f454d46a741e4414ef3586d/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:34:27.568823 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a7cfd-1734-4b0b-96b3-a3fe640f7892 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:27.578809 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a7cfd-1734-4b0b-96b3-a3fe640f7892 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:27.609076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:27.634172 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a3139-abce-4d36-8361-1f29adb8e4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:27.634778 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a3139-abce-4d36-8361-1f29adb8e4c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:27.635063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a3139-abce-4d36-8361-1f29adb8e4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:27.635340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a3139-abce-4d36-8361-1f29adb8e4c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:27.635978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a3139-abce-4d36-8361-1f29adb8e4c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:27.637559 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7a3139-abce-4d36-8361-1f29adb8e4c6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:27.648228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1115/2234] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:34:27.652442 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a0c5e-714a-4ff3-acff-9ce7610a034d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:27.663863 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a0c5e-714a-4ff3-acff-9ce7610a034d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:27.668693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3dd09-bfc1-4b4e-8139-dc6681112ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:27.669288 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3dd09-bfc1-4b4e-8139-dc6681112ee1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:27.669571 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3dd09-bfc1-4b4e-8139-dc6681112ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:27.669848 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3dd09-bfc1-4b4e-8139-dc6681112ee1 admin admin] PATH_INFO: `/v3/projects/b964110459f54020ae3963e8aba796f4/users/3ca05db84f454d46a741e4414ef3586d/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:27.671814 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3dd09-bfc1-4b4e-8139-dc6681112ee1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b964110459f54020ae3963e8aba796f4, user_id=3ca05db84f454d46a741e4414ef3586d, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:27.674490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3dd09-bfc1-4b4e-8139-dc6681112ee1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:27.684791 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1120/2235] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:34:27 2022] PUT /identity/v3/projects/b964110459f54020ae3963e8aba796f4/users/3ca05db84f454d46a741e4414ef3586d/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:27.701653 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63db66-1522-463f-b9c6-bfe3e38b2b49 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:27.701932 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63db66-1522-463f-b9c6-bfe3e38b2b49 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:27.702203 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63db66-1522-463f-b9c6-bfe3e38b2b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:27.731145 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1d63db66-1522-463f-b9c6-bfe3e38b2b49 None None] MFA Rules not processed for user `3ca05db84f454d46a741e4414ef3586d`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:27.732433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1d63db66-1522-463f-b9c6-bfe3e38b2b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:27.800595 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1116/2236] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:27 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:27.811946 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3dd09-bfc1-4b4e-8139-dc6681112ee1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:27.813685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-25a3dd09-bfc1-4b4e-8139-dc6681112ee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:27.871553 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3dd09-bfc1-4b4e-8139-dc6681112ee1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:27.874989 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19763603-85bd-4f7f-baeb-2981211bd6ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:27.875850 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19763603-85bd-4f7f-baeb-2981211bd6ec service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:27.876126 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19763603-85bd-4f7f-baeb-2981211bd6ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:27.876513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19763603-85bd-4f7f-baeb-2981211bd6ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:27.886518 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19763603-85bd-4f7f-baeb-2981211bd6ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:27.887968 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19763603-85bd-4f7f-baeb-2981211bd6ec service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:27.897110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1121/2237] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:34:27 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:34:28.093959 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63db66-1522-463f-b9c6-bfe3e38b2b49 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:28.103746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63db66-1522-463f-b9c6-bfe3e38b2b49 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:28.107394 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b43a8-cfb8-41ef-82c2-5b6483e8825a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:28.107991 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b43a8-cfb8-41ef-82c2-5b6483e8825a service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:28.108274 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b43a8-cfb8-41ef-82c2-5b6483e8825a service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:28.108547 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b43a8-cfb8-41ef-82c2-5b6483e8825a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:28.166096 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b43a8-cfb8-41ef-82c2-5b6483e8825a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:28.167629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b43a8-cfb8-41ef-82c2-5b6483e8825a service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:28.177059 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1117/2238] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:34:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:34:32.549313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19763603-85bd-4f7f-baeb-2981211bd6ec service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:32.551024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-19763603-85bd-4f7f-baeb-2981211bd6ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:32.621230 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19763603-85bd-4f7f-baeb-2981211bd6ec service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:32.626051 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac29f3a-d93c-49c7-9462-97ec97758e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:32.626687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac29f3a-d93c-49c7-9462-97ec97758e09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:32.626997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac29f3a-d93c-49c7-9462-97ec97758e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:32.627274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac29f3a-d93c-49c7-9462-97ec97758e09 admin admin] PATH_INFO: `/v3/users/a4b8c1479d3d4905b777f060a76c5cb2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:32.628406 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac29f3a-d93c-49c7-9462-97ec97758e09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4b8c1479d3d4905b777f060a76c5cb2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:32.630326 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac29f3a-d93c-49c7-9462-97ec97758e09 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:32.830581 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-eac29f3a-d93c-49c7-9462-97ec97758e09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4b8c1479d3d4905b777f060a76c5cb2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:32.837627 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-eac29f3a-d93c-49c7-9462-97ec97758e09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4b8c1479d3d4905b777f060a76c5cb2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:32.902317 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-eac29f3a-d93c-49c7-9462-97ec97758e09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4b8c1479d3d4905b777f060a76c5cb2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:32.903032 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-eac29f3a-d93c-49c7-9462-97ec97758e09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4b8c1479d3d4905b777f060a76c5cb2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:32.908182 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:32.916723 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-eac29f3a-d93c-49c7-9462-97ec97758e09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4b8c1479d3d4905b777f060a76c5cb2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:32.922954 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1122/2239] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:34:32 2022] DELETE /identity/v3/users/a4b8c1479d3d4905b777f060a76c5cb2 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:32.927959 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb386190-ec5b-4ce8-9f13-d18a6f38c379 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:32.928502 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb386190-ec5b-4ce8-9f13-d18a6f38c379 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:32.928585 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb386190-ec5b-4ce8-9f13-d18a6f38c379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:32.949252 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-cb386190-ec5b-4ce8-9f13-d18a6f38c379 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:32.950465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cb386190-ec5b-4ce8-9f13-d18a6f38c379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:32.999330 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:33.012011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1118/2240] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:33.021600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac29f3a-d93c-49c7-9462-97ec97758e09 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:33.023039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-eac29f3a-d93c-49c7-9462-97ec97758e09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:33.086370 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac29f3a-d93c-49c7-9462-97ec97758e09 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:33.089832 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf4574-b8c1-486f-a349-921443699202 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:33.090491 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf4574-b8c1-486f-a349-921443699202 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:33.090796 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf4574-b8c1-486f-a349-921443699202 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:33.091056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf4574-b8c1-486f-a349-921443699202 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:33.100631 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf4574-b8c1-486f-a349-921443699202 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:33.102042 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf4574-b8c1-486f-a349-921443699202 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:33.112716 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1123/2241] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:34:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:34:33.366018 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb386190-ec5b-4ce8-9f13-d18a6f38c379 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:33.367534 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cb386190-ec5b-4ce8-9f13-d18a6f38c379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:33.384182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb386190-ec5b-4ce8-9f13-d18a6f38c379 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:33.388250 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f3b082-a12f-4a39-ac3e-d907e101f05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:33.389176 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3b082-a12f-4a39-ac3e-d907e101f05a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:33.389176 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3b082-a12f-4a39-ac3e-d907e101f05a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:33.389448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f3b082-a12f-4a39-ac3e-d907e101f05a admin admin] PATH_INFO: `/v3/users/300cfe69b06a4d83a8aad0eb73df9408` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:33.390603 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3b082-a12f-4a39-ac3e-d907e101f05a admin admin] RBAC: Authorizing `identity:delete_user(user_id=300cfe69b06a4d83a8aad0eb73df9408)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:33.392561 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f3b082-a12f-4a39-ac3e-d907e101f05a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:33.562163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf4574-b8c1-486f-a349-921443699202 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:33.563681 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c9bf4574-b8c1-486f-a349-921443699202 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:33.578114 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf4574-b8c1-486f-a349-921443699202 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:33.585085 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdafe63-d320-4f5c-92b3-7468217fe31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:33.586110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdafe63-d320-4f5c-92b3-7468217fe31a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:33.587227 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdafe63-d320-4f5c-92b3-7468217fe31a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:33.587673 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdafe63-d320-4f5c-92b3-7468217fe31a admin admin] PATH_INFO: `/v3/projects/633915e61f6e4acaa0de4ff3861d8526` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:33.589228 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdafe63-d320-4f5c-92b3-7468217fe31a admin admin] RBAC: Authorizing `identity:delete_project(project_id=633915e61f6e4acaa0de4ff3861d8526)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:33.592553 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdafe63-d320-4f5c-92b3-7468217fe31a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:33.603718 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-64f3b082-a12f-4a39-ac3e-d907e101f05a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '300cfe69b06a4d83a8aad0eb73df9408'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:33.692102 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-64f3b082-a12f-4a39-ac3e-d907e101f05a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '300cfe69b06a4d83a8aad0eb73df9408'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:33.692723 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-64f3b082-a12f-4a39-ac3e-d907e101f05a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '300cfe69b06a4d83a8aad0eb73df9408'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:33.693480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-64f3b082-a12f-4a39-ac3e-d907e101f05a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '300cfe69b06a4d83a8aad0eb73df9408'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:33.708710 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-64f3b082-a12f-4a39-ac3e-d907e101f05a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '300cfe69b06a4d83a8aad0eb73df9408'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:33.724757 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1119/2242] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:34:33 2022] DELETE /identity/v3/users/300cfe69b06a4d83a8aad0eb73df9408 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:33.729850 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9254eb-aa8b-4be7-916b-dae4adec90dd None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:33.730577 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9254eb-aa8b-4be7-916b-dae4adec90dd None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:33.734277 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9254eb-aa8b-4be7-916b-dae4adec90dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:33.735834 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cbdafe63-d320-4f5c-92b3-7468217fe31a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '633915e61f6e4acaa0de4ff3861d8526', 'request_id': 'req-cbdafe63-d320-4f5c-92b3-7468217fe31a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:33.755766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9a9254eb-aa8b-4be7-916b-dae4adec90dd None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:33.757046 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9a9254eb-aa8b-4be7-916b-dae4adec90dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:33.800552 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cbdafe63-d320-4f5c-92b3-7468217fe31a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '633915e61f6e4acaa0de4ff3861d8526', 'request_id': 'req-cbdafe63-d320-4f5c-92b3-7468217fe31a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:33.808304 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cbdafe63-d320-4f5c-92b3-7468217fe31a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '633915e61f6e4acaa0de4ff3861d8526', 'request_id': 'req-cbdafe63-d320-4f5c-92b3-7468217fe31a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:33.824381 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cbdafe63-d320-4f5c-92b3-7468217fe31a admin admin] The token cache is being invalidate because project 633915e61f6e4acaa0de4ff3861d8526 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:34:33.824657 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cbdafe63-d320-4f5c-92b3-7468217fe31a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:33.825304 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cbdafe63-d320-4f5c-92b3-7468217fe31a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:33.829414 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1124/2244] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:34:33 2022] DELETE /identity/v3/projects/633915e61f6e4acaa0de4ff3861d8526 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:34:33.829535 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1120/2244] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:34:33.832471 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77094c7-49a0-43d7-92f8-69aa3a85ad47 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:33.832835 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77094c7-49a0-43d7-92f8-69aa3a85ad47 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:33.833174 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77094c7-49a0-43d7-92f8-69aa3a85ad47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:33.837340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9254eb-aa8b-4be7-916b-dae4adec90dd None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:33.838718 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9a9254eb-aa8b-4be7-916b-dae4adec90dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:33.862731 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e77094c7-49a0-43d7-92f8-69aa3a85ad47 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:33.862985 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e77094c7-49a0-43d7-92f8-69aa3a85ad47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:33.876900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1125/2245] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:33.886453 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fddbc-3bb4-494d-9b73-8873133086d3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:33.886841 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fddbc-3bb4-494d-9b73-8873133086d3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:33.887313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fddbc-3bb4-494d-9b73-8873133086d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:33.907502 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-791fddbc-3bb4-494d-9b73-8873133086d3 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:33.908726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-791fddbc-3bb4-494d-9b73-8873133086d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:33.920211 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1126/2246] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:33.920347 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9254eb-aa8b-4be7-916b-dae4adec90dd None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:33.922759 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791fddbc-3bb4-494d-9b73-8873133086d3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:33.923116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfa5a3-cf58-46e5-816f-1909c8af25e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:33.923801 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfa5a3-cf58-46e5-816f-1909c8af25e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:33.924149 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfa5a3-cf58-46e5-816f-1909c8af25e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:33.924547 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfa5a3-cf58-46e5-816f-1909c8af25e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:33.926395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-96dfa5a3-cf58-46e5-816f-1909c8af25e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:33.932165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791fddbc-3bb4-494d-9b73-8873133086d3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:33.936986 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c3e8c-cacf-431d-8bc8-0b49a54b0f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:33.937619 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c3e8c-cacf-431d-8bc8-0b49a54b0f5c admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:33.937930 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c3e8c-cacf-431d-8bc8-0b49a54b0f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:33.938343 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815c3e8c-cacf-431d-8bc8-0b49a54b0f5c admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:33.939046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c3e8c-cacf-431d-8bc8-0b49a54b0f5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:34:33.939352 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c3e8c-cacf-431d-8bc8-0b49a54b0f5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:33.939526 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dfa5a3-cf58-46e5-816f-1909c8af25e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:33.941263 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815c3e8c-cacf-431d-8bc8-0b49a54b0f5c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:33.944221 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dfa5a3-cf58-46e5-816f-1909c8af25e9 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:33.953893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1127/2247] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:34:33.955081 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1121/2248] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:34:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:34:33.968122 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfa5a3-cf58-46e5-816f-1909c8af25e9 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:33.982109 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfa5a3-cf58-46e5-816f-1909c8af25e9 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:33.987541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991a15f-7813-4d6d-9167-23a98621990f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:33.988158 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991a15f-7813-4d6d-9167-23a98621990f admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:33.988440 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991a15f-7813-4d6d-9167-23a98621990f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:33.988709 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991a15f-7813-4d6d-9167-23a98621990f admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:33.989345 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991a15f-7813-4d6d-9167-23a98621990f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:33.991303 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991a15f-7813-4d6d-9167-23a98621990f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:33.999045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:34.162935 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1122/2249] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:34:33 2022] POST /identity/v3/projects => generated 379 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:34:34.164297 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c3e8c-cacf-431d-8bc8-0b49a54b0f5c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:34.181677 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815c3e8c-cacf-431d-8bc8-0b49a54b0f5c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:34.253334 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:34.278470 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5a0bd-6404-4d67-8e13-d3ed30ca14eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:34.279189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5a0bd-6404-4d67-8e13-d3ed30ca14eb admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:34.279473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5a0bd-6404-4d67-8e13-d3ed30ca14eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:34.279747 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5a0bd-6404-4d67-8e13-d3ed30ca14eb admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:34.280416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5a0bd-6404-4d67-8e13-d3ed30ca14eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:34.282406 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5a0bd-6404-4d67-8e13-d3ed30ca14eb admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:34.398650 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1128/2250] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:34:34 2022] POST /identity/v3/users => generated 290 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:34:34.400892 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991a15f-7813-4d6d-9167-23a98621990f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:34.419726 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991a15f-7813-4d6d-9167-23a98621990f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:34.426426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221eb10-7e11-48a6-8d1d-d170000b1336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:34.427077 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221eb10-7e11-48a6-8d1d-d170000b1336 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:34.427357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221eb10-7e11-48a6-8d1d-d170000b1336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:34.428225 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221eb10-7e11-48a6-8d1d-d170000b1336 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:34.428584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221eb10-7e11-48a6-8d1d-d170000b1336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:34.430073 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221eb10-7e11-48a6-8d1d-d170000b1336 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:34.444007 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1123/2251] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:34:34.452903 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5a0bd-6404-4d67-8e13-d3ed30ca14eb admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:34.473686 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5a0bd-6404-4d67-8e13-d3ed30ca14eb admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:34.482010 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c27b0-53dd-4588-ae1f-7566116ecb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:34.482883 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c27b0-53dd-4588-ae1f-7566116ecb0c admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:34.483301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c27b0-53dd-4588-ae1f-7566116ecb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:34.483859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c27b0-53dd-4588-ae1f-7566116ecb0c admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:34.484836 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743c27b0-53dd-4588-ae1f-7566116ecb0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:34.486912 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743c27b0-53dd-4588-ae1f-7566116ecb0c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:34.506510 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1129/2252] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:34:34.508625 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221eb10-7e11-48a6-8d1d-d170000b1336 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:34.532508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221eb10-7e11-48a6-8d1d-d170000b1336 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:34.539619 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86efd8-29ab-428d-a641-10cfa0efdf43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:34.540888 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86efd8-29ab-428d-a641-10cfa0efdf43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:34.541266 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86efd8-29ab-428d-a641-10cfa0efdf43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:34.541674 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86efd8-29ab-428d-a641-10cfa0efdf43 admin admin] PATH_INFO: `/v3/projects/2c5938d62e2d40969d096b66e14edd75/users/d935a4a30e4d44e0a392596452a5c1b6/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:34.563910 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86efd8-29ab-428d-a641-10cfa0efdf43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c5938d62e2d40969d096b66e14edd75, user_id=d935a4a30e4d44e0a392596452a5c1b6, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:34.566733 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86efd8-29ab-428d-a641-10cfa0efdf43 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:34.598036 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c27b0-53dd-4588-ae1f-7566116ecb0c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:34.600160 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-743c27b0-53dd-4588-ae1f-7566116ecb0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:34.631081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c27b0-53dd-4588-ae1f-7566116ecb0c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:34.651717 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5b6ba-1a36-44c8-9a5d-c16879038952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:34.652460 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5b6ba-1a36-44c8-9a5d-c16879038952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:34.653075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5b6ba-1a36-44c8-9a5d-c16879038952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:34.653432 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5b6ba-1a36-44c8-9a5d-c16879038952 admin admin] PATH_INFO: `/v3/projects/5795af30e2d04e1f87ba8a5dda346824` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:34.654649 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5b6ba-1a36-44c8-9a5d-c16879038952 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5795af30e2d04e1f87ba8a5dda346824)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:34.657294 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5b6ba-1a36-44c8-9a5d-c16879038952 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:34.691625 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1124/2253] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:34:34 2022] PUT /identity/v3/projects/2c5938d62e2d40969d096b66e14edd75/users/d935a4a30e4d44e0a392596452a5c1b6/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:34.698428 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d67b32-801a-4aea-a9ac-56b72bfa40f0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:34.698734 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d67b32-801a-4aea-a9ac-56b72bfa40f0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:34.699009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d67b32-801a-4aea-a9ac-56b72bfa40f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:34.719902 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-84d67b32-801a-4aea-a9ac-56b72bfa40f0 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:34.724470 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-84d67b32-801a-4aea-a9ac-56b72bfa40f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:34.808503 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1125/2254] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:34:34.819202 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d67b32-801a-4aea-a9ac-56b72bfa40f0 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:34.819971 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c7a5b6ba-1a36-44c8-9a5d-c16879038952 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5795af30e2d04e1f87ba8a5dda346824', 'request_id': 'req-c7a5b6ba-1a36-44c8-9a5d-c16879038952'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:34.820608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-84d67b32-801a-4aea-a9ac-56b72bfa40f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:34.948018 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d67b32-801a-4aea-a9ac-56b72bfa40f0 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:34.948018 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d81c5af-a5ca-476e-a0dd-c837c82819b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:34.948018 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81c5af-a5ca-476e-a0dd-c837c82819b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:34.948018 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81c5af-a5ca-476e-a0dd-c837c82819b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:34.949158 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d81c5af-a5ca-476e-a0dd-c837c82819b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:34.966308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:35.010470 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d81c5af-a5ca-476e-a0dd-c837c82819b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:35.012135 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d81c5af-a5ca-476e-a0dd-c837c82819b4 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:35.022037 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1126/2255] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:34:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:34:35.103676 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c7a5b6ba-1a36-44c8-9a5d-c16879038952 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5795af30e2d04e1f87ba8a5dda346824', 'request_id': 'req-c7a5b6ba-1a36-44c8-9a5d-c16879038952'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:35.109861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c7a5b6ba-1a36-44c8-9a5d-c16879038952 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5795af30e2d04e1f87ba8a5dda346824', 'request_id': 'req-c7a5b6ba-1a36-44c8-9a5d-c16879038952'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:35.121864 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c7a5b6ba-1a36-44c8-9a5d-c16879038952 admin admin] The token cache is being invalidate because project 5795af30e2d04e1f87ba8a5dda346824 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:34:35.122146 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c7a5b6ba-1a36-44c8-9a5d-c16879038952 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:35.122841 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c7a5b6ba-1a36-44c8-9a5d-c16879038952 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:35.124513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1130/2256] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:34:34 2022] DELETE /identity/v3/projects/5795af30e2d04e1f87ba8a5dda346824 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:34:35.128879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d81c5af-a5ca-476e-a0dd-c837c82819b4 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:35.140363 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d81c5af-a5ca-476e-a0dd-c837c82819b4 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:35.145226 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632fbf45-ccd9-42ad-a92a-27a0bada5bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:35.145881 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fbf45-ccd9-42ad-a92a-27a0bada5bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:35.146155 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fbf45-ccd9-42ad-a92a-27a0bada5bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:35.146424 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fbf45-ccd9-42ad-a92a-27a0bada5bcc admin admin] PATH_INFO: `/v3/users/a54b51e13cfc4a0e965fd52c9ff9b82c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:35.147610 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632fbf45-ccd9-42ad-a92a-27a0bada5bcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a54b51e13cfc4a0e965fd52c9ff9b82c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:35.149498 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632fbf45-ccd9-42ad-a92a-27a0bada5bcc admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:35.475375 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-632fbf45-ccd9-42ad-a92a-27a0bada5bcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a54b51e13cfc4a0e965fd52c9ff9b82c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:35.553816 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-632fbf45-ccd9-42ad-a92a-27a0bada5bcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a54b51e13cfc4a0e965fd52c9ff9b82c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:35.554620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-632fbf45-ccd9-42ad-a92a-27a0bada5bcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a54b51e13cfc4a0e965fd52c9ff9b82c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:35.555846 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-632fbf45-ccd9-42ad-a92a-27a0bada5bcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a54b51e13cfc4a0e965fd52c9ff9b82c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:35.569512 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-632fbf45-ccd9-42ad-a92a-27a0bada5bcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a54b51e13cfc4a0e965fd52c9ff9b82c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:35.582137 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1127/2257] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:34:35 2022] DELETE /identity/v3/users/a54b51e13cfc4a0e965fd52c9ff9b82c => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:36.145143 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5b6ba-1a36-44c8-9a5d-c16879038952 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:36.145143 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c7a5b6ba-1a36-44c8-9a5d-c16879038952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:36.154980 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5b6ba-1a36-44c8-9a5d-c16879038952 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:36.214031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a1b208-f41b-4f7f-a142-e5cccff39906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:36.214889 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1b208-f41b-4f7f-a142-e5cccff39906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:36.215177 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1b208-f41b-4f7f-a142-e5cccff39906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:36.215471 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1b208-f41b-4f7f-a142-e5cccff39906 admin admin] PATH_INFO: `/v3/projects/cbdce41fbf784aa2951c9c6362106c9f` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:36.216643 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a1b208-f41b-4f7f-a142-e5cccff39906 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbdce41fbf784aa2951c9c6362106c9f)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:36.218575 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a1b208-f41b-4f7f-a142-e5cccff39906 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:36.320125 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=85,keystone:INSERT=4,keystone:DELETE=11 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:36.321395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-43a1b208-f41b-4f7f-a142-e5cccff39906 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbdce41fbf784aa2951c9c6362106c9f', 'request_id': 'req-43a1b208-f41b-4f7f-a142-e5cccff39906'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:36.411671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-43a1b208-f41b-4f7f-a142-e5cccff39906 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbdce41fbf784aa2951c9c6362106c9f', 'request_id': 'req-43a1b208-f41b-4f7f-a142-e5cccff39906'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:36.419152 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-43a1b208-f41b-4f7f-a142-e5cccff39906 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbdce41fbf784aa2951c9c6362106c9f', 'request_id': 'req-43a1b208-f41b-4f7f-a142-e5cccff39906'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:36.430986 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-43a1b208-f41b-4f7f-a142-e5cccff39906 admin admin] The token cache is being invalidate because project cbdce41fbf784aa2951c9c6362106c9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:34:36.431274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-43a1b208-f41b-4f7f-a142-e5cccff39906 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:36.431941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-43a1b208-f41b-4f7f-a142-e5cccff39906 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:36.433429 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1131/2258] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:34:36 2022] DELETE /identity/v3/projects/cbdce41fbf784aa2951c9c6362106c9f => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:36.438477 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01f226-e627-4986-bd01-e1ce2c92955d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:36.438994 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01f226-e627-4986-bd01-e1ce2c92955d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:36.439177 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01f226-e627-4986-bd01-e1ce2c92955d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:36.460533 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7e01f226-e627-4986-bd01-e1ce2c92955d None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:36.462299 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7e01f226-e627-4986-bd01-e1ce2c92955d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:36.528291 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1128/2259] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:36.536063 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fff55-cbe2-4efb-b02d-09d0a710bb96 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:36.536424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fff55-cbe2-4efb-b02d-09d0a710bb96 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:36.536767 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fff55-cbe2-4efb-b02d-09d0a710bb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:36.557750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-809fff55-cbe2-4efb-b02d-09d0a710bb96 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:36.559054 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-809fff55-cbe2-4efb-b02d-09d0a710bb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:36.567621 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1132/2260] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:34:36.572035 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01f226-e627-4986-bd01-e1ce2c92955d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:36.582823 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01f226-e627-4986-bd01-e1ce2c92955d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:36.602898 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b765223-bb66-47d3-bd88-85eaa3fd3a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:36.603620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b765223-bb66-47d3-bd88-85eaa3fd3a87 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:36.603967 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b765223-bb66-47d3-bd88-85eaa3fd3a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:36.604363 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b765223-bb66-47d3-bd88-85eaa3fd3a87 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:36.605094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b765223-bb66-47d3-bd88-85eaa3fd3a87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:34:36.605403 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b765223-bb66-47d3-bd88-85eaa3fd3a87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:36.608867 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b765223-bb66-47d3-bd88-85eaa3fd3a87 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:36.626769 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1129/2261] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:34:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:34:36.631557 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809fff55-cbe2-4efb-b02d-09d0a710bb96 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:36.652778 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809fff55-cbe2-4efb-b02d-09d0a710bb96 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:36.659260 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b643b5-6b86-407c-bbbb-d766aaa786c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:36.659944 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b643b5-6b86-407c-bbbb-d766aaa786c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:36.660294 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b643b5-6b86-407c-bbbb-d766aaa786c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:36.660693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b643b5-6b86-407c-bbbb-d766aaa786c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:36.661392 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b643b5-6b86-407c-bbbb-d766aaa786c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:36.665477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b643b5-6b86-407c-bbbb-d766aaa786c3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:36.770745 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1133/2262] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:34:36 2022] POST /identity/v3/projects => generated 379 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:34:36.772653 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b765223-bb66-47d3-bd88-85eaa3fd3a87 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:36.784166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b765223-bb66-47d3-bd88-85eaa3fd3a87 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:36.797603 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:36.852395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fdedeb-0861-4c6e-8b4a-fe5a740e16e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:36.854445 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fdedeb-0861-4c6e-8b4a-fe5a740e16e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:36.854445 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fdedeb-0861-4c6e-8b4a-fe5a740e16e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:36.854445 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fdedeb-0861-4c6e-8b4a-fe5a740e16e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:36.854696 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fdedeb-0861-4c6e-8b4a-fe5a740e16e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:36.856959 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fdedeb-0861-4c6e-8b4a-fe5a740e16e2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:36.962162 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1130/2263] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:34:36 2022] POST /identity/v3/users => generated 290 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:34:36.964531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b643b5-6b86-407c-bbbb-d766aaa786c3 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:36.974935 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b643b5-6b86-407c-bbbb-d766aaa786c3 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:36.980060 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1eee8-b556-4b04-9091-74ec7c7f6f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:36.982970 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1eee8-b556-4b04-9091-74ec7c7f6f8e admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:36.982970 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1eee8-b556-4b04-9091-74ec7c7f6f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:36.982970 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb1eee8-b556-4b04-9091-74ec7c7f6f8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:36.982970 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb1eee8-b556-4b04-9091-74ec7c7f6f8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:36.983647 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb1eee8-b556-4b04-9091-74ec7c7f6f8e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:36.997755 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1134/2264] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:34:36.999244 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fdedeb-0861-4c6e-8b4a-fe5a740e16e2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:37.013733 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fdedeb-0861-4c6e-8b4a-fe5a740e16e2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:37.019396 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e25a54b-c46c-435c-b58b-c0cc9863f2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:37.020268 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25a54b-c46c-435c-b58b-c0cc9863f2df admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:37.020620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25a54b-c46c-435c-b58b-c0cc9863f2df admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:37.021563 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25a54b-c46c-435c-b58b-c0cc9863f2df admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:37.021695 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e25a54b-c46c-435c-b58b-c0cc9863f2df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:37.023371 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e25a54b-c46c-435c-b58b-c0cc9863f2df admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:37.046322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1131/2265] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:34:37.050495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1eee8-b556-4b04-9091-74ec7c7f6f8e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:37.062376 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb1eee8-b556-4b04-9091-74ec7c7f6f8e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:37.068846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f52288-325f-47bc-8172-c24d9e86dd78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:37.069542 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f52288-325f-47bc-8172-c24d9e86dd78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:37.069822 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f52288-325f-47bc-8172-c24d9e86dd78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:37.070097 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f52288-325f-47bc-8172-c24d9e86dd78 admin admin] PATH_INFO: `/v3/projects/4c960c919939439ebdf267286db3c0d3/users/ea29be4ff4e54c0e94706de46c5e2659/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:37.094111 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f52288-325f-47bc-8172-c24d9e86dd78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c960c919939439ebdf267286db3c0d3, user_id=ea29be4ff4e54c0e94706de46c5e2659, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:37.097163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f52288-325f-47bc-8172-c24d9e86dd78 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:37.178277 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1135/2266] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:34:37 2022] PUT /identity/v3/projects/4c960c919939439ebdf267286db3c0d3/users/ea29be4ff4e54c0e94706de46c5e2659/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:34:37.191195 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf4f67-32f9-4330-bed4-1b23a1281ff9 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:37.191480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf4f67-32f9-4330-bed4-1b23a1281ff9 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:37.191755 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf4f67-32f9-4330-bed4-1b23a1281ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:37.220095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-dcbf4f67-32f9-4330-bed4-1b23a1281ff9 None None] MFA Rules not processed for user `ea29be4ff4e54c0e94706de46c5e2659`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:37.221396 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-dcbf4f67-32f9-4330-bed4-1b23a1281ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:37.296636 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1132/2267] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:37.297109 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f52288-325f-47bc-8172-c24d9e86dd78 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:37.393624 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f52288-325f-47bc-8172-c24d9e86dd78 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:37.398790 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312fed9-9454-4a28-be59-94363198909a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:37.402984 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312fed9-9454-4a28-be59-94363198909a admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:37.402984 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312fed9-9454-4a28-be59-94363198909a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:37.402984 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312fed9-9454-4a28-be59-94363198909a admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:37.402984 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312fed9-9454-4a28-be59-94363198909a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:37.402984 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312fed9-9454-4a28-be59-94363198909a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:37.489968 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1136/2268] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:34:37 2022] POST /identity/v3/projects => generated 387 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:34:37.497632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf4f67-32f9-4330-bed4-1b23a1281ff9 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:37.512189 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf4f67-32f9-4330-bed4-1b23a1281ff9 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:37.596689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c70c3-3023-421e-9934-f174a0e10790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:37.597674 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c70c3-3023-421e-9934-f174a0e10790 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:37.598171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c70c3-3023-421e-9934-f174a0e10790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:37.599075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c70c3-3023-421e-9934-f174a0e10790 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:37.599842 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27c70c3-3023-421e-9934-f174a0e10790 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:37.602766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27c70c3-3023-421e-9934-f174a0e10790 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:37.738829 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1133/2269] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:34:37 2022] POST /identity/v3/users => generated 300 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:34:37.744606 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312fed9-9454-4a28-be59-94363198909a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:37.755664 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312fed9-9454-4a28-be59-94363198909a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:37.760352 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3045a-910a-4db7-9627-45f7a4da17f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:37.760954 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3045a-910a-4db7-9627-45f7a4da17f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:37.761235 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3045a-910a-4db7-9627-45f7a4da17f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:37.761509 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3045a-910a-4db7-9627-45f7a4da17f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:37.762161 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3045a-910a-4db7-9627-45f7a4da17f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:37.763776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd3045a-910a-4db7-9627-45f7a4da17f0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:37.776812 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1137/2270] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:34:37.780986 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c70c3-3023-421e-9934-f174a0e10790 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:37.791390 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c70c3-3023-421e-9934-f174a0e10790 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:37.796591 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f09f1-adf0-454e-9930-6e0260ae4c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:37.797247 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f09f1-adf0-454e-9930-6e0260ae4c0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:37.797526 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f09f1-adf0-454e-9930-6e0260ae4c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:37.797795 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3f09f1-adf0-454e-9930-6e0260ae4c0f admin admin] PATH_INFO: `/v3/projects/78dfa42ab0854b0dbab0a97e3971e400/users/34b37cc0d5c341d7b9a8116c92583d43/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:37.812763 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f09f1-adf0-454e-9930-6e0260ae4c0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=78dfa42ab0854b0dbab0a97e3971e400, user_id=34b37cc0d5c341d7b9a8116c92583d43, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:37.815354 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3f09f1-adf0-454e-9930-6e0260ae4c0f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:37.901688 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1134/2271] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:34:37 2022] PUT /identity/v3/projects/78dfa42ab0854b0dbab0a97e3971e400/users/34b37cc0d5c341d7b9a8116c92583d43/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:37.903287 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3045a-910a-4db7-9627-45f7a4da17f0 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:37.914923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3045a-910a-4db7-9627-45f7a4da17f0 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:37.988091 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:38.001717 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291951ed-b06a-4c5a-bbe6-e288a0fcaef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:38.002341 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291951ed-b06a-4c5a-bbe6-e288a0fcaef5 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:38.002613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291951ed-b06a-4c5a-bbe6-e288a0fcaef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:38.002950 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291951ed-b06a-4c5a-bbe6-e288a0fcaef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:38.003604 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291951ed-b06a-4c5a-bbe6-e288a0fcaef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:38.005352 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291951ed-b06a-4c5a-bbe6-e288a0fcaef5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:38.016721 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1138/2272] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:34:38.020947 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f09f1-adf0-454e-9930-6e0260ae4c0f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:38.032851 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3f09f1-adf0-454e-9930-6e0260ae4c0f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:38.040321 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d76542-0c08-4a3e-9bd2-7ac64fb1a326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:38.040321 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d76542-0c08-4a3e-9bd2-7ac64fb1a326 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:38.040690 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d76542-0c08-4a3e-9bd2-7ac64fb1a326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:38.040898 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d76542-0c08-4a3e-9bd2-7ac64fb1a326 admin admin] PATH_INFO: `/v3/projects/78dfa42ab0854b0dbab0a97e3971e400/users/34b37cc0d5c341d7b9a8116c92583d43/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:38.043066 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d76542-0c08-4a3e-9bd2-7ac64fb1a326 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78dfa42ab0854b0dbab0a97e3971e400, user_id=34b37cc0d5c341d7b9a8116c92583d43, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:38.046229 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d76542-0c08-4a3e-9bd2-7ac64fb1a326 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:38.060861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1135/2273] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:34:38 2022] PUT /identity/v3/projects/78dfa42ab0854b0dbab0a97e3971e400/users/34b37cc0d5c341d7b9a8116c92583d43/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:34:38.071715 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96f597-1dfd-485d-a852-bf3d6c1cb6cb None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:38.072226 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96f597-1dfd-485d-a852-bf3d6c1cb6cb None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:38.072494 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96f597-1dfd-485d-a852-bf3d6c1cb6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:38.105166 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ad96f597-1dfd-485d-a852-bf3d6c1cb6cb None None] MFA Rules not processed for user `34b37cc0d5c341d7b9a8116c92583d43`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:38.106147 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ad96f597-1dfd-485d-a852-bf3d6c1cb6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:38.143464 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:38.180618 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1139/2274] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:38 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:34:38.193232 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d76542-0c08-4a3e-9bd2-7ac64fb1a326 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:38.197450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-63d76542-0c08-4a3e-9bd2-7ac64fb1a326 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:38.275686 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d76542-0c08-4a3e-9bd2-7ac64fb1a326 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:38.280231 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed7d5a-48a5-4080-8c85-93a6813bfdeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:38.281058 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed7d5a-48a5-4080-8c85-93a6813bfdeb service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:38.281495 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed7d5a-48a5-4080-8c85-93a6813bfdeb service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:38.281912 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed7d5a-48a5-4080-8c85-93a6813bfdeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:38.294319 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed7d5a-48a5-4080-8c85-93a6813bfdeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:38.296122 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed7d5a-48a5-4080-8c85-93a6813bfdeb service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:38.307412 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1136/2275] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:34:38 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:34:38.393914 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96f597-1dfd-485d-a852-bf3d6c1cb6cb None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:38.464950 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96f597-1dfd-485d-a852-bf3d6c1cb6cb None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:38.472275 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f9fe5-dba0-4613-bb4b-9bb0f4c5f624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:38.473339 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f9fe5-dba0-4613-bb4b-9bb0f4c5f624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:38.474370 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f9fe5-dba0-4613-bb4b-9bb0f4c5f624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:38.474543 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f9fe5-dba0-4613-bb4b-9bb0f4c5f624 admin admin] PATH_INFO: `/v3/users/ea29be4ff4e54c0e94706de46c5e2659` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:38.476363 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4f9fe5-dba0-4613-bb4b-9bb0f4c5f624 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea29be4ff4e54c0e94706de46c5e2659)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:38.479067 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4f9fe5-dba0-4613-bb4b-9bb0f4c5f624 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:38.662129 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7f4f9fe5-dba0-4613-bb4b-9bb0f4c5f624 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea29be4ff4e54c0e94706de46c5e2659'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:38.669555 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7f4f9fe5-dba0-4613-bb4b-9bb0f4c5f624 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea29be4ff4e54c0e94706de46c5e2659'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:38.772386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7f4f9fe5-dba0-4613-bb4b-9bb0f4c5f624 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea29be4ff4e54c0e94706de46c5e2659'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:38.773377 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7f4f9fe5-dba0-4613-bb4b-9bb0f4c5f624 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea29be4ff4e54c0e94706de46c5e2659'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:38.792052 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7f4f9fe5-dba0-4613-bb4b-9bb0f4c5f624 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea29be4ff4e54c0e94706de46c5e2659'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:38.799349 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1140/2276] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:34:38 2022] DELETE /identity/v3/users/ea29be4ff4e54c0e94706de46c5e2659 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:38.804227 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a7aa8-10e1-4b7e-bb95-2785f07d57c4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:38.804690 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a7aa8-10e1-4b7e-bb95-2785f07d57c4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:38.804864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a7aa8-10e1-4b7e-bb95-2785f07d57c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:38.825354 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-710a7aa8-10e1-4b7e-bb95-2785f07d57c4 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:38.826566 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-710a7aa8-10e1-4b7e-bb95-2785f07d57c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:38.869788 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:38.891922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1137/2277] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:38.904095 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f9fe5-dba0-4613-bb4b-9bb0f4c5f624 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:38.906131 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7f4f9fe5-dba0-4613-bb4b-9bb0f4c5f624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:38.972149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f9fe5-dba0-4613-bb4b-9bb0f4c5f624 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:38.976755 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7431a-0311-4efa-942c-2a3e8501ebe1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:38.977304 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7431a-0311-4efa-942c-2a3e8501ebe1 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:38.977585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7431a-0311-4efa-942c-2a3e8501ebe1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:38.977855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7431a-0311-4efa-942c-2a3e8501ebe1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:38.988290 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7431a-0311-4efa-942c-2a3e8501ebe1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:38.991065 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7431a-0311-4efa-942c-2a3e8501ebe1 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:39.000746 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1141/2278] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:34:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:34:39.795778 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a7aa8-10e1-4b7e-bb95-2785f07d57c4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:39.799775 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-710a7aa8-10e1-4b7e-bb95-2785f07d57c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:39.809538 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a7aa8-10e1-4b7e-bb95-2785f07d57c4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:39.814445 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef9f62-0044-440f-9aa0-6b8e7c03c30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:39.816531 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef9f62-0044-440f-9aa0-6b8e7c03c30c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:39.816531 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef9f62-0044-440f-9aa0-6b8e7c03c30c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:39.816531 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ef9f62-0044-440f-9aa0-6b8e7c03c30c admin admin] PATH_INFO: `/v3/projects/4c960c919939439ebdf267286db3c0d3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:39.817792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ef9f62-0044-440f-9aa0-6b8e7c03c30c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c960c919939439ebdf267286db3c0d3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:39.819789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ef9f62-0044-440f-9aa0-6b8e7c03c30c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:39.984648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f3ef9f62-0044-440f-9aa0-6b8e7c03c30c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c960c919939439ebdf267286db3c0d3', 'request_id': 'req-f3ef9f62-0044-440f-9aa0-6b8e7c03c30c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:39.992586 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f3ef9f62-0044-440f-9aa0-6b8e7c03c30c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c960c919939439ebdf267286db3c0d3', 'request_id': 'req-f3ef9f62-0044-440f-9aa0-6b8e7c03c30c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:40.000270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f3ef9f62-0044-440f-9aa0-6b8e7c03c30c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c960c919939439ebdf267286db3c0d3', 'request_id': 'req-f3ef9f62-0044-440f-9aa0-6b8e7c03c30c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:40.096726 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f3ef9f62-0044-440f-9aa0-6b8e7c03c30c admin admin] The token cache is being invalidate because project 4c960c919939439ebdf267286db3c0d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:34:40.097071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f3ef9f62-0044-440f-9aa0-6b8e7c03c30c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:40.102855 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f3ef9f62-0044-440f-9aa0-6b8e7c03c30c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:40.102855 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1138/2279] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:34:39 2022] DELETE /identity/v3/projects/4c960c919939439ebdf267286db3c0d3 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:40.104095 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7431a-0311-4efa-942c-2a3e8501ebe1 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:40.114288 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7431a-0311-4efa-942c-2a3e8501ebe1 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:40.177604 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9f200-5da9-4175-914c-d21509704131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:40.178540 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9f200-5da9-4175-914c-d21509704131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:40.178658 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9f200-5da9-4175-914c-d21509704131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:40.179019 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9f200-5da9-4175-914c-d21509704131 admin admin] PATH_INFO: `/v3/users/34b37cc0d5c341d7b9a8116c92583d43` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:40.181220 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b9f200-5da9-4175-914c-d21509704131 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34b37cc0d5c341d7b9a8116c92583d43)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:40.183985 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b9f200-5da9-4175-914c-d21509704131 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:40.526902 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-94b9f200-5da9-4175-914c-d21509704131 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34b37cc0d5c341d7b9a8116c92583d43'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:40.534556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-94b9f200-5da9-4175-914c-d21509704131 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34b37cc0d5c341d7b9a8116c92583d43'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:40.622805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-94b9f200-5da9-4175-914c-d21509704131 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34b37cc0d5c341d7b9a8116c92583d43'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:40.625112 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-94b9f200-5da9-4175-914c-d21509704131 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34b37cc0d5c341d7b9a8116c92583d43'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:40.644724 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-94b9f200-5da9-4175-914c-d21509704131 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34b37cc0d5c341d7b9a8116c92583d43'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:40.655270 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1142/2280] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:34:40 2022] DELETE /identity/v3/users/34b37cc0d5c341d7b9a8116c92583d43 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:41.180708 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef9f62-0044-440f-9aa0-6b8e7c03c30c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:41.182197 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f3ef9f62-0044-440f-9aa0-6b8e7c03c30c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:41.204452 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:41.216417 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ef9f62-0044-440f-9aa0-6b8e7c03c30c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:41.307713 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a431fd-bd0c-496a-b8ad-79bfed95d140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:41.311005 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a431fd-bd0c-496a-b8ad-79bfed95d140 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:41.311637 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a431fd-bd0c-496a-b8ad-79bfed95d140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:41.312024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a431fd-bd0c-496a-b8ad-79bfed95d140 admin admin] PATH_INFO: `/v3/projects/78dfa42ab0854b0dbab0a97e3971e400` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:41.313470 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a431fd-bd0c-496a-b8ad-79bfed95d140 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78dfa42ab0854b0dbab0a97e3971e400)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:41.315618 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a431fd-bd0c-496a-b8ad-79bfed95d140 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:41.326753 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d71252-0c6d-4500-acb6-a923c39b322d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:41.327212 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d71252-0c6d-4500-acb6-a923c39b322d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:41.327861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d71252-0c6d-4500-acb6-a923c39b322d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:41.353561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:41.364147 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-50d71252-0c6d-4500-acb6-a923c39b322d None None] MFA Rules not processed for user `d935a4a30e4d44e0a392596452a5c1b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:41.365547 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-50d71252-0c6d-4500-acb6-a923c39b322d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:41.451599 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1143/2281] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:34:41.457737 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c4a431fd-bd0c-496a-b8ad-79bfed95d140 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78dfa42ab0854b0dbab0a97e3971e400', 'request_id': 'req-c4a431fd-bd0c-496a-b8ad-79bfed95d140'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:41.458026 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d71252-0c6d-4500-acb6-a923c39b322d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:41.459531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-50d71252-0c6d-4500-acb6-a923c39b322d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:41.466121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c4a431fd-bd0c-496a-b8ad-79bfed95d140 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78dfa42ab0854b0dbab0a97e3971e400', 'request_id': 'req-c4a431fd-bd0c-496a-b8ad-79bfed95d140'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:41.472730 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c4a431fd-bd0c-496a-b8ad-79bfed95d140 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78dfa42ab0854b0dbab0a97e3971e400', 'request_id': 'req-c4a431fd-bd0c-496a-b8ad-79bfed95d140'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:41.475476 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d71252-0c6d-4500-acb6-a923c39b322d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:41.546385 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad829a9-7dbc-4df7-ad3c-52d126ca7ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:41.548702 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad829a9-7dbc-4df7-ad3c-52d126ca7ad5 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:41.548702 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad829a9-7dbc-4df7-ad3c-52d126ca7ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:41.548702 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad829a9-7dbc-4df7-ad3c-52d126ca7ad5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:41.548702 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad829a9-7dbc-4df7-ad3c-52d126ca7ad5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:41.550957 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad829a9-7dbc-4df7-ad3c-52d126ca7ad5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:41.552053 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c4a431fd-bd0c-496a-b8ad-79bfed95d140 admin admin] The token cache is being invalidate because project 78dfa42ab0854b0dbab0a97e3971e400 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:34:41.552420 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c4a431fd-bd0c-496a-b8ad-79bfed95d140 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:41.553144 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c4a431fd-bd0c-496a-b8ad-79bfed95d140 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:41.559005 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1139/2282] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:34:41 2022] DELETE /identity/v3/projects/78dfa42ab0854b0dbab0a97e3971e400 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:41.560610 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3a94b-d5b9-4f4b-990d-ecbc48326547 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:41.560963 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3a94b-d5b9-4f4b-990d-ecbc48326547 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:41.561305 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3a94b-d5b9-4f4b-990d-ecbc48326547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:41.582998 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2fc3a94b-d5b9-4f4b-990d-ecbc48326547 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:41.584215 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2fc3a94b-d5b9-4f4b-990d-ecbc48326547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:41.593197 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1140/2283] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:41.600696 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ba0bc-da55-436f-b7e9-5787dad5f95a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:41.601073 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ba0bc-da55-436f-b7e9-5787dad5f95a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:41.601413 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ba0bc-da55-436f-b7e9-5787dad5f95a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:41.621289 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a33ba0bc-da55-436f-b7e9-5787dad5f95a None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:41.622571 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a33ba0bc-da55-436f-b7e9-5787dad5f95a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:41.631179 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1141/2284] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:34:41.635369 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ba0bc-da55-436f-b7e9-5787dad5f95a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:41.644498 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ba0bc-da55-436f-b7e9-5787dad5f95a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:41.649662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb90c43-f082-42e1-a65c-5973591fa83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:41.650250 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb90c43-f082-42e1-a65c-5973591fa83e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:41.650537 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb90c43-f082-42e1-a65c-5973591fa83e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:41.650855 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb90c43-f082-42e1-a65c-5973591fa83e admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:41.651563 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb90c43-f082-42e1-a65c-5973591fa83e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:34:41.651803 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb90c43-f082-42e1-a65c-5973591fa83e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:41.653383 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb90c43-f082-42e1-a65c-5973591fa83e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:41.662766 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1144/2285] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:34:41 2022] POST /identity/v3/projects => generated 379 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:34:41.663938 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad829a9-7dbc-4df7-ad3c-52d126ca7ad5 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:41.669316 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1142/2286] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:34:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:34:41.669457 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-dad829a9-7dbc-4df7-ad3c-52d126ca7ad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:41.670354 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb90c43-f082-42e1-a65c-5973591fa83e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:41.681889 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb90c43-f082-42e1-a65c-5973591fa83e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:41.733815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad829a9-7dbc-4df7-ad3c-52d126ca7ad5 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:41.738544 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad42468-3dd9-4121-8807-e4d03a134f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:41.739189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad42468-3dd9-4121-8807-e4d03a134f2e admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:41.739476 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad42468-3dd9-4121-8807-e4d03a134f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:41.739829 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad42468-3dd9-4121-8807-e4d03a134f2e admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:41.740388 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad42468-3dd9-4121-8807-e4d03a134f2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:41.742326 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad42468-3dd9-4121-8807-e4d03a134f2e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:41.742435 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75622678-9b9d-476a-9940-87d188b634fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:41.743063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75622678-9b9d-476a-9940-87d188b634fd admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:41.743335 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75622678-9b9d-476a-9940-87d188b634fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:41.743605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75622678-9b9d-476a-9940-87d188b634fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:41.745081 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75622678-9b9d-476a-9940-87d188b634fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:41.747377 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75622678-9b9d-476a-9940-87d188b634fd admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:41.764142 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:41.854758 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1145/2287] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:34:41 2022] POST /identity/v3/users => generated 296 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:34:41.857166 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad42468-3dd9-4121-8807-e4d03a134f2e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:41.867159 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad42468-3dd9-4121-8807-e4d03a134f2e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:41.875386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33851239-9a66-45e9-95a8-30e801a69480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:41.876181 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33851239-9a66-45e9-95a8-30e801a69480 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:41.876308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33851239-9a66-45e9-95a8-30e801a69480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:41.876525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33851239-9a66-45e9-95a8-30e801a69480 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:41.877261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33851239-9a66-45e9-95a8-30e801a69480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:41.879122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33851239-9a66-45e9-95a8-30e801a69480 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:41.893991 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1143/2288] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:34:41 2022] POST /identity/v3/projects => generated 383 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:34:41.894217 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1146/2289] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:34:41.896445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33851239-9a66-45e9-95a8-30e801a69480 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:41.896545 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75622678-9b9d-476a-9940-87d188b634fd admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:41.902912 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:41.908277 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33851239-9a66-45e9-95a8-30e801a69480 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:41.979940 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75622678-9b9d-476a-9940-87d188b634fd admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:41.983262 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46efd7-0ee2-413a-88b4-016c25a8d981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:41.983983 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46efd7-0ee2-413a-88b4-016c25a8d981 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:41.984212 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9bc77-442b-4461-a78a-978c7a88a84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:41.984462 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46efd7-0ee2-413a-88b4-016c25a8d981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:41.984831 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d46efd7-0ee2-413a-88b4-016c25a8d981 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:41.984949 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9bc77-442b-4461-a78a-978c7a88a84b admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:41.985147 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9bc77-442b-4461-a78a-978c7a88a84b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:41.985600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9bc77-442b-4461-a78a-978c7a88a84b admin admin] PATH_INFO: `/v3/projects/bd528e84c5b24fe19e0f7fa9365c943a/users/2ab56d1b8a804202bcea64a1d0105f99/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:41.985935 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d46efd7-0ee2-413a-88b4-016c25a8d981 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:41.988023 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d46efd7-0ee2-413a-88b4-016c25a8d981 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:42.003404 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c9bc77-442b-4461-a78a-978c7a88a84b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd528e84c5b24fe19e0f7fa9365c943a, user_id=2ab56d1b8a804202bcea64a1d0105f99, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:42.006484 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c9bc77-442b-4461-a78a-978c7a88a84b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:42.092472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1147/2290] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:34:41 2022] PUT /identity/v3/projects/bd528e84c5b24fe19e0f7fa9365c943a/users/2ab56d1b8a804202bcea64a1d0105f99/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:42.099233 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9bc77-442b-4461-a78a-978c7a88a84b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:42.100022 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1144/2291] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:34:41 2022] POST /identity/v3/users => generated 292 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:34:42.104274 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46efd7-0ee2-413a-88b4-016c25a8d981 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:42.116032 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d46efd7-0ee2-413a-88b4-016c25a8d981 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:42.170170 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9bc77-442b-4461-a78a-978c7a88a84b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:42.175280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a6be8-db84-4277-9fee-510aff47ac3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:42.176055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a6be8-db84-4277-9fee-510aff47ac3b admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:42.176542 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a6be8-db84-4277-9fee-510aff47ac3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:42.177239 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a6be8-db84-4277-9fee-510aff47ac3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:42.177413 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1429161-0ebe-4c35-a125-bed784f79a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:42.177628 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1429161-0ebe-4c35-a125-bed784f79a76 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:42.177945 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a6be8-db84-4277-9fee-510aff47ac3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:42.178036 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1429161-0ebe-4c35-a125-bed784f79a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:42.178318 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1429161-0ebe-4c35-a125-bed784f79a76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:42.179712 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4a6be8-db84-4277-9fee-510aff47ac3b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:42.180452 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1429161-0ebe-4c35-a125-bed784f79a76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:42.182137 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1429161-0ebe-4c35-a125-bed784f79a76 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:42.193230 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1148/2292] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:34:42.195580 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1145/2293] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:34:42.198609 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a6be8-db84-4277-9fee-510aff47ac3b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:42.206988 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1429161-0ebe-4c35-a125-bed784f79a76 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:42.212695 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a6be8-db84-4277-9fee-510aff47ac3b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:42.219652 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fed82-9c44-491d-bcf2-7c25b5a4b2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:42.220333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fed82-9c44-491d-bcf2-7c25b5a4b2b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:42.220690 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1429161-0ebe-4c35-a125-bed784f79a76 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:42.220813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fed82-9c44-491d-bcf2-7c25b5a4b2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:42.221151 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fed82-9c44-491d-bcf2-7c25b5a4b2b5 admin admin] PATH_INFO: `/v3/projects/bd528e84c5b24fe19e0f7fa9365c943a/users/2ab56d1b8a804202bcea64a1d0105f99/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:42.223818 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fed82-9c44-491d-bcf2-7c25b5a4b2b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd528e84c5b24fe19e0f7fa9365c943a, user_id=2ab56d1b8a804202bcea64a1d0105f99, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:42.226611 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fed82-9c44-491d-bcf2-7c25b5a4b2b5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:42.232292 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670998d-83de-4806-a232-03e4eef74527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:42.237470 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670998d-83de-4806-a232-03e4eef74527 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:42.237470 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670998d-83de-4806-a232-03e4eef74527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:42.237470 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670998d-83de-4806-a232-03e4eef74527 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:42.237470 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c670998d-83de-4806-a232-03e4eef74527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:42.237470 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c670998d-83de-4806-a232-03e4eef74527 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:42.253207 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1149/2294] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:34:42 2022] PUT /identity/v3/projects/bd528e84c5b24fe19e0f7fa9365c943a/users/2ab56d1b8a804202bcea64a1d0105f99/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:42.256160 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1146/2295] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:34:42.263777 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fed82-9c44-491d-bcf2-7c25b5a4b2b5 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:42.330269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fed82-9c44-491d-bcf2-7c25b5a4b2b5 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:42.336558 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16813bc-44ef-4545-a4f3-dd21d5c0a74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:42.337402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16813bc-44ef-4545-a4f3-dd21d5c0a74d admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:42.337689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16813bc-44ef-4545-a4f3-dd21d5c0a74d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:42.337964 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16813bc-44ef-4545-a4f3-dd21d5c0a74d admin admin] PATH_INFO: `/v3/projects/e909518e56ca454b8fbd120daa8e7e8e/users/72ab7ad1bf5d484494b087365e410dca/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:42.350464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:42.354250 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16813bc-44ef-4545-a4f3-dd21d5c0a74d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e909518e56ca454b8fbd120daa8e7e8e, user_id=72ab7ad1bf5d484494b087365e410dca, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:42.356927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16813bc-44ef-4545-a4f3-dd21d5c0a74d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:42.480929 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1150/2296] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:34:42 2022] PUT /identity/v3/projects/e909518e56ca454b8fbd120daa8e7e8e/users/72ab7ad1bf5d484494b087365e410dca/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:34:42.492384 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03222c3-a649-4fac-8712-b1b526d72e09 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:42.492776 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03222c3-a649-4fac-8712-b1b526d72e09 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:42.493055 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03222c3-a649-4fac-8712-b1b526d72e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:42.523016 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e03222c3-a649-4fac-8712-b1b526d72e09 None None] MFA Rules not processed for user `72ab7ad1bf5d484494b087365e410dca`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:42.524759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e03222c3-a649-4fac-8712-b1b526d72e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:42.597631 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:42.603446 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1147/2297] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:42.608359 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16813bc-44ef-4545-a4f3-dd21d5c0a74d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:42.682111 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16813bc-44ef-4545-a4f3-dd21d5c0a74d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:42.687000 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542cb77-f7bb-46f5-8f89-7ada7c43e5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:42.687530 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542cb77-f7bb-46f5-8f89-7ada7c43e5df admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:42.687822 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542cb77-f7bb-46f5-8f89-7ada7c43e5df admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:42.688099 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542cb77-f7bb-46f5-8f89-7ada7c43e5df admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:42.688744 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542cb77-f7bb-46f5-8f89-7ada7c43e5df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:42.690654 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542cb77-f7bb-46f5-8f89-7ada7c43e5df admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:42.829934 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1151/2298] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:34:42 2022] POST /identity/v3/projects => generated 385 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:34:42.834074 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03222c3-a649-4fac-8712-b1b526d72e09 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:42.872490 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03222c3-a649-4fac-8712-b1b526d72e09 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:42.942289 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3795de5-cecb-4508-818a-eaea82648091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:42.942954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3795de5-cecb-4508-818a-eaea82648091 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:42.943239 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3795de5-cecb-4508-818a-eaea82648091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:42.943513 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3795de5-cecb-4508-818a-eaea82648091 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:42.944163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3795de5-cecb-4508-818a-eaea82648091 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:42.946201 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3795de5-cecb-4508-818a-eaea82648091 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:43.091305 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1148/2299] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:34:42 2022] POST /identity/v3/users => generated 299 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:34:43.091885 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542cb77-f7bb-46f5-8f89-7ada7c43e5df admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:43.106600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542cb77-f7bb-46f5-8f89-7ada7c43e5df admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:43.125529 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ef4cc-40cb-4d31-bc4f-6b1e19671522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:43.125529 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ef4cc-40cb-4d31-bc4f-6b1e19671522 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:43.125529 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ef4cc-40cb-4d31-bc4f-6b1e19671522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:43.125529 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ef4cc-40cb-4d31-bc4f-6b1e19671522 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:43.126077 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4ef4cc-40cb-4d31-bc4f-6b1e19671522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:43.126655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4ef4cc-40cb-4d31-bc4f-6b1e19671522 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:43.140650 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1152/2300] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:43 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:34:43.145151 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3795de5-cecb-4508-818a-eaea82648091 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:43.158970 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3795de5-cecb-4508-818a-eaea82648091 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:43.173334 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e286d15-dd6c-4b58-80d5-ae6b1e1f713f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:43.174265 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e286d15-dd6c-4b58-80d5-ae6b1e1f713f admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:43.176019 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e286d15-dd6c-4b58-80d5-ae6b1e1f713f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:43.176556 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e286d15-dd6c-4b58-80d5-ae6b1e1f713f admin admin] PATH_INFO: `/v3/projects/4ec9aade863042379abbdfa76c30b1b7/users/76d7a675bcfd4c21bc6da196f21647f1/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:43.203781 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e286d15-dd6c-4b58-80d5-ae6b1e1f713f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ec9aade863042379abbdfa76c30b1b7, user_id=76d7a675bcfd4c21bc6da196f21647f1, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:43.206512 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e286d15-dd6c-4b58-80d5-ae6b1e1f713f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:43.289729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1149/2301] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:34:43 2022] PUT /identity/v3/projects/4ec9aade863042379abbdfa76c30b1b7/users/76d7a675bcfd4c21bc6da196f21647f1/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:43.293549 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ef4cc-40cb-4d31-bc4f-6b1e19671522 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:43.309773 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ef4cc-40cb-4d31-bc4f-6b1e19671522 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:43.393901 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57eb39a-16b5-46f4-8535-d62d7fb03d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:43.394595 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57eb39a-16b5-46f4-8535-d62d7fb03d77 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:43.394888 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57eb39a-16b5-46f4-8535-d62d7fb03d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:43.395170 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57eb39a-16b5-46f4-8535-d62d7fb03d77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:43.395823 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57eb39a-16b5-46f4-8535-d62d7fb03d77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:43.397444 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57eb39a-16b5-46f4-8535-d62d7fb03d77 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:43.409716 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1153/2302] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:43 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:34:43.412518 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e286d15-dd6c-4b58-80d5-ae6b1e1f713f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:43.423706 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e286d15-dd6c-4b58-80d5-ae6b1e1f713f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:43.428456 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0295d30-3a18-4c0f-8376-f9717e718a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:43.429109 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0295d30-3a18-4c0f-8376-f9717e718a57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:43.429392 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0295d30-3a18-4c0f-8376-f9717e718a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:43.429667 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0295d30-3a18-4c0f-8376-f9717e718a57 admin admin] PATH_INFO: `/v3/projects/4ec9aade863042379abbdfa76c30b1b7/users/76d7a675bcfd4c21bc6da196f21647f1/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:43.432207 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0295d30-3a18-4c0f-8376-f9717e718a57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ec9aade863042379abbdfa76c30b1b7, user_id=76d7a675bcfd4c21bc6da196f21647f1, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:43.435024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0295d30-3a18-4c0f-8376-f9717e718a57 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:43.445635 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1150/2303] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:34:43 2022] PUT /identity/v3/projects/4ec9aade863042379abbdfa76c30b1b7/users/76d7a675bcfd4c21bc6da196f21647f1/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:43.459819 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387a63f-82f4-4448-a102-7fe9c86d8a7c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:43.460135 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387a63f-82f4-4448-a102-7fe9c86d8a7c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:43.460507 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387a63f-82f4-4448-a102-7fe9c86d8a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:43.505064 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1387a63f-82f4-4448-a102-7fe9c86d8a7c None None] MFA Rules not processed for user `76d7a675bcfd4c21bc6da196f21647f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:43.505064 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1387a63f-82f4-4448-a102-7fe9c86d8a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:43.556979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:43.590777 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1154/2304] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:43.598503 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0295d30-3a18-4c0f-8376-f9717e718a57 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:43.600492 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c0295d30-3a18-4c0f-8376-f9717e718a57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:43.682912 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0295d30-3a18-4c0f-8376-f9717e718a57 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:43.686816 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ef41b-4ee4-4f81-8c21-8cf1396e1fe3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:43.687443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ef41b-4ee4-4f81-8c21-8cf1396e1fe3 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:43.687767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ef41b-4ee4-4f81-8c21-8cf1396e1fe3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:43.687995 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284ef41b-4ee4-4f81-8c21-8cf1396e1fe3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:43.763169 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ef41b-4ee4-4f81-8c21-8cf1396e1fe3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:43.764812 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284ef41b-4ee4-4f81-8c21-8cf1396e1fe3 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:43.775971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1151/2305] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:34:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:34:48.532029 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2994e11-cb43-4403-a10d-1881f9726059 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:48.532597 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2994e11-cb43-4403-a10d-1881f9726059 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:48.532597 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2994e11-cb43-4403-a10d-1881f9726059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:48.567432 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b2994e11-cb43-4403-a10d-1881f9726059 None None] MFA Rules not processed for user `2ab56d1b8a804202bcea64a1d0105f99`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:48.568690 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b2994e11-cb43-4403-a10d-1881f9726059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:48.650476 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1155/2306] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:48 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:48.666248 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ef41b-4ee4-4f81-8c21-8cf1396e1fe3 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:48.668455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-284ef41b-4ee4-4f81-8c21-8cf1396e1fe3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:48.679121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:48.789913 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284ef41b-4ee4-4f81-8c21-8cf1396e1fe3 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:48.797208 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d9bedf-ef92-463d-915b-3564af553223 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:48.797844 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9bedf-ef92-463d-915b-3564af553223 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:48.798276 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9bedf-ef92-463d-915b-3564af553223 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:48.798567 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9bedf-ef92-463d-915b-3564af553223 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:48.801645 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c2d9bedf-ef92-463d-915b-3564af553223 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:48.868505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9bedf-ef92-463d-915b-3564af553223 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:48.869544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9bedf-ef92-463d-915b-3564af553223 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:48.879819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1152/2307] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:34:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:34:51.740923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2994e11-cb43-4403-a10d-1881f9726059 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:51.742452 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b2994e11-cb43-4403-a10d-1881f9726059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:51.830942 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2994e11-cb43-4403-a10d-1881f9726059 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:51.844169 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7293e-2394-42d1-bd7b-801f21e52c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:51.845469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7293e-2394-42d1-bd7b-801f21e52c0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:51.845887 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7293e-2394-42d1-bd7b-801f21e52c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:51.846258 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7293e-2394-42d1-bd7b-801f21e52c0f admin admin] PATH_INFO: `/v3/users/d21a777bb5264f1990e3cc2a73837b99` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:51.847945 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a7293e-2394-42d1-bd7b-801f21e52c0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d21a777bb5264f1990e3cc2a73837b99)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:51.850149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a7293e-2394-42d1-bd7b-801f21e52c0f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:52.111739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-75a7293e-2394-42d1-bd7b-801f21e52c0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd21a777bb5264f1990e3cc2a73837b99'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:52.125548 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-75a7293e-2394-42d1-bd7b-801f21e52c0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd21a777bb5264f1990e3cc2a73837b99'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:52.251434 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-75a7293e-2394-42d1-bd7b-801f21e52c0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd21a777bb5264f1990e3cc2a73837b99'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:52.267001 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-75a7293e-2394-42d1-bd7b-801f21e52c0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd21a777bb5264f1990e3cc2a73837b99'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:52.267001 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:52.270291 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-75a7293e-2394-42d1-bd7b-801f21e52c0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd21a777bb5264f1990e3cc2a73837b99'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:52.277878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1156/2308] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:34:51 2022] DELETE /identity/v3/users/d21a777bb5264f1990e3cc2a73837b99 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:52.282584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214dd86-0a12-4479-ab3f-637ae02697eb None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:52.283165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214dd86-0a12-4479-ab3f-637ae02697eb None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:52.283165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1214dd86-0a12-4479-ab3f-637ae02697eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:52.304966 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1214dd86-0a12-4479-ab3f-637ae02697eb None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:52.306354 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1214dd86-0a12-4479-ab3f-637ae02697eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:52.392974 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1153/2309] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:52.403228 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7293e-2394-42d1-bd7b-801f21e52c0f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:52.405006 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-75a7293e-2394-42d1-bd7b-801f21e52c0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:52.468885 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7293e-2394-42d1-bd7b-801f21e52c0f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:52.472965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed16a6f0-3dbc-4b28-ab2a-daff767db507 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:52.473682 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16a6f0-3dbc-4b28-ab2a-daff767db507 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:52.473975 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16a6f0-3dbc-4b28-ab2a-daff767db507 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:52.474245 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed16a6f0-3dbc-4b28-ab2a-daff767db507 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:52.484522 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed16a6f0-3dbc-4b28-ab2a-daff767db507 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:52.486015 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed16a6f0-3dbc-4b28-ab2a-daff767db507 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:52.497337 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1157/2310] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:34:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:34:53.225854 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214dd86-0a12-4479-ab3f-637ae02697eb None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:53.227446 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1214dd86-0a12-4479-ab3f-637ae02697eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:53.241334 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1214dd86-0a12-4479-ab3f-637ae02697eb None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:53.251014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c06036-ae58-4b29-8840-b0d3441eb473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:53.252109 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c06036-ae58-4b29-8840-b0d3441eb473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:53.252477 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c06036-ae58-4b29-8840-b0d3441eb473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:53.252822 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c06036-ae58-4b29-8840-b0d3441eb473 admin admin] PATH_INFO: `/v3/projects/d380d80d37b943f695988ebc32eb39a3` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:53.254200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c06036-ae58-4b29-8840-b0d3441eb473 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d380d80d37b943f695988ebc32eb39a3)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:53.256663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c06036-ae58-4b29-8840-b0d3441eb473 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:53.351120 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:53.364979 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-94c06036-ae58-4b29-8840-b0d3441eb473 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd380d80d37b943f695988ebc32eb39a3', 'request_id': 'req-94c06036-ae58-4b29-8840-b0d3441eb473'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:53.371466 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-94c06036-ae58-4b29-8840-b0d3441eb473 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd380d80d37b943f695988ebc32eb39a3', 'request_id': 'req-94c06036-ae58-4b29-8840-b0d3441eb473'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:53.379422 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-94c06036-ae58-4b29-8840-b0d3441eb473 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd380d80d37b943f695988ebc32eb39a3', 'request_id': 'req-94c06036-ae58-4b29-8840-b0d3441eb473'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:53.444859 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-94c06036-ae58-4b29-8840-b0d3441eb473 admin admin] The token cache is being invalidate because project d380d80d37b943f695988ebc32eb39a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:34:53.446087 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-94c06036-ae58-4b29-8840-b0d3441eb473 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:53.446740 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-94c06036-ae58-4b29-8840-b0d3441eb473 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:53.448255 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1154/2311] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:34:53 2022] DELETE /identity/v3/projects/d380d80d37b943f695988ebc32eb39a3 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:53.453106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162d91c-619f-4875-b99f-2c43db97960e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:53.453604 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162d91c-619f-4875-b99f-2c43db97960e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:53.453712 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162d91c-619f-4875-b99f-2c43db97960e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:53.475535 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5162d91c-619f-4875-b99f-2c43db97960e None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:53.476783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5162d91c-619f-4875-b99f-2c43db97960e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:53.547247 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1158/2312] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:53.557020 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d204d-1b0f-4230-9582-e6ed85bd514a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:53.557332 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d204d-1b0f-4230-9582-e6ed85bd514a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:53.557687 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d204d-1b0f-4230-9582-e6ed85bd514a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:53.580069 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6f2d204d-1b0f-4230-9582-e6ed85bd514a None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:53.581416 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6f2d204d-1b0f-4230-9582-e6ed85bd514a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:53.590108 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1155/2313] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:53.594600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162d91c-619f-4875-b99f-2c43db97960e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:53.605784 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162d91c-619f-4875-b99f-2c43db97960e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:53.610878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e49e3-8af7-4649-8a53-2525bed3de2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:53.611650 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e49e3-8af7-4649-8a53-2525bed3de2a admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:53.612060 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e49e3-8af7-4649-8a53-2525bed3de2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:53.612265 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e49e3-8af7-4649-8a53-2525bed3de2a admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:53.612925 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87e49e3-8af7-4649-8a53-2525bed3de2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:34:53.613227 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87e49e3-8af7-4649-8a53-2525bed3de2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:53.616290 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87e49e3-8af7-4649-8a53-2525bed3de2a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:53.630128 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1159/2314] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:34:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:34:53.635797 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d204d-1b0f-4230-9582-e6ed85bd514a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:53.648500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d204d-1b0f-4230-9582-e6ed85bd514a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:53.653585 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb939b-03d9-4efc-a8bb-881ad1af1b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:53.657629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb939b-03d9-4efc-a8bb-881ad1af1b15 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:53.657629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb939b-03d9-4efc-a8bb-881ad1af1b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:53.657629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb939b-03d9-4efc-a8bb-881ad1af1b15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:53.657629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb939b-03d9-4efc-a8bb-881ad1af1b15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:53.657629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb939b-03d9-4efc-a8bb-881ad1af1b15 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:53.748283 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1156/2315] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:34:53 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:34:53.749684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e49e3-8af7-4649-8a53-2525bed3de2a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:53.762469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e49e3-8af7-4649-8a53-2525bed3de2a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:53.828485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db055504-70f3-4d53-a636-9dae5bbdffeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:53.829090 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db055504-70f3-4d53-a636-9dae5bbdffeb admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:53.829367 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db055504-70f3-4d53-a636-9dae5bbdffeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:53.829637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db055504-70f3-4d53-a636-9dae5bbdffeb admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:53.830270 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db055504-70f3-4d53-a636-9dae5bbdffeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:53.832264 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db055504-70f3-4d53-a636-9dae5bbdffeb admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:53.955480 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1160/2316] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:34:53 2022] POST /identity/v3/users => generated 287 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:34:53.958491 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb939b-03d9-4efc-a8bb-881ad1af1b15 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:53.969007 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb939b-03d9-4efc-a8bb-881ad1af1b15 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:53.973601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c8b1a-cce3-4741-a72c-a28fea11eec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:53.974225 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c8b1a-cce3-4741-a72c-a28fea11eec0 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:53.974502 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c8b1a-cce3-4741-a72c-a28fea11eec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:53.974803 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c8b1a-cce3-4741-a72c-a28fea11eec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:53.975457 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c8b1a-cce3-4741-a72c-a28fea11eec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:53.977582 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c8b1a-cce3-4741-a72c-a28fea11eec0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:53.988206 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1157/2317] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:34:53.992937 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db055504-70f3-4d53-a636-9dae5bbdffeb admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:54.006522 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db055504-70f3-4d53-a636-9dae5bbdffeb admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:54.011628 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c01bf-f4c2-49fa-a88c-61889221353b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:54.012236 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c01bf-f4c2-49fa-a88c-61889221353b admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:54.012678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c01bf-f4c2-49fa-a88c-61889221353b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:54.013173 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c01bf-f4c2-49fa-a88c-61889221353b admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:54.014299 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c01bf-f4c2-49fa-a88c-61889221353b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:54.017422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c01bf-f4c2-49fa-a88c-61889221353b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:54.024106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:54.029760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1161/2318] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:34:54.033742 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c8b1a-cce3-4741-a72c-a28fea11eec0 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:54.044281 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c8b1a-cce3-4741-a72c-a28fea11eec0 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:54.052754 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d21c38-ae27-4919-a972-485ab7a4bc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:54.053510 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d21c38-ae27-4919-a972-485ab7a4bc2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:54.053904 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d21c38-ae27-4919-a972-485ab7a4bc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:54.058275 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d21c38-ae27-4919-a972-485ab7a4bc2c admin admin] PATH_INFO: `/v3/projects/64574f0e703b42629d5948259c02dd84/users/d5dffea476a248d9820fa6739cbdc910/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:54.075006 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d21c38-ae27-4919-a972-485ab7a4bc2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=64574f0e703b42629d5948259c02dd84, user_id=d5dffea476a248d9820fa6739cbdc910, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:54.077861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d21c38-ae27-4919-a972-485ab7a4bc2c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:54.159323 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1158/2319] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:34:54 2022] PUT /identity/v3/projects/64574f0e703b42629d5948259c02dd84/users/d5dffea476a248d9820fa6739cbdc910/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:54.170904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43528364-aadd-4848-a799-f4057f5dc6c4 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:54.172544 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43528364-aadd-4848-a799-f4057f5dc6c4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:54.172544 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43528364-aadd-4848-a799-f4057f5dc6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:54.199319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-43528364-aadd-4848-a799-f4057f5dc6c4 None None] MFA Rules not processed for user `d5dffea476a248d9820fa6739cbdc910`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:54.206966 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-43528364-aadd-4848-a799-f4057f5dc6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:54.267616 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1162/2320] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:34:54.280079 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d21c38-ae27-4919-a972-485ab7a4bc2c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:54.281759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-29d21c38-ae27-4919-a972-485ab7a4bc2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:54.344368 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d21c38-ae27-4919-a972-485ab7a4bc2c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:54.349136 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1362b325-8349-4419-91ff-5c9837c62033 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:54.349798 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362b325-8349-4419-91ff-5c9837c62033 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:54.350079 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362b325-8349-4419-91ff-5c9837c62033 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:54.350354 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1362b325-8349-4419-91ff-5c9837c62033 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:54.360277 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1362b325-8349-4419-91ff-5c9837c62033 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:54.361722 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1362b325-8349-4419-91ff-5c9837c62033 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:54.370867 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1159/2321] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:34:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:34:54.395879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43528364-aadd-4848-a799-f4057f5dc6c4 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:54.458619 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43528364-aadd-4848-a799-f4057f5dc6c4 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:54.463789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92cb7e4-678c-4d3a-bc33-1de6446de5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:54.464420 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92cb7e4-678c-4d3a-bc33-1de6446de5b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:54.464703 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92cb7e4-678c-4d3a-bc33-1de6446de5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:54.464977 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92cb7e4-678c-4d3a-bc33-1de6446de5b8 admin admin] PATH_INFO: `/v3/users/d5dffea476a248d9820fa6739cbdc910` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:54.466108 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92cb7e4-678c-4d3a-bc33-1de6446de5b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5dffea476a248d9820fa6739cbdc910)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:54.468065 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92cb7e4-678c-4d3a-bc33-1de6446de5b8 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:54.649337 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b92cb7e4-678c-4d3a-bc33-1de6446de5b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5dffea476a248d9820fa6739cbdc910'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:54.657291 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b92cb7e4-678c-4d3a-bc33-1de6446de5b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5dffea476a248d9820fa6739cbdc910'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:54.719441 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b92cb7e4-678c-4d3a-bc33-1de6446de5b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5dffea476a248d9820fa6739cbdc910'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:54.720093 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b92cb7e4-678c-4d3a-bc33-1de6446de5b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5dffea476a248d9820fa6739cbdc910'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:54.732711 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b92cb7e4-678c-4d3a-bc33-1de6446de5b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5dffea476a248d9820fa6739cbdc910'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:54.738804 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1163/2322] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:34:54 2022] DELETE /identity/v3/users/d5dffea476a248d9820fa6739cbdc910 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:34:54.743428 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b2075-0784-4fef-b908-4c7355e6ad31 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:54.743721 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b2075-0784-4fef-b908-4c7355e6ad31 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:54.743996 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b2075-0784-4fef-b908-4c7355e6ad31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:54.765441 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-da4b2075-0784-4fef-b908-4c7355e6ad31 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:54.766644 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-da4b2075-0784-4fef-b908-4c7355e6ad31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:54.835125 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1160/2323] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:54.846416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92cb7e4-678c-4d3a-bc33-1de6446de5b8 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:54.849202 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b92cb7e4-678c-4d3a-bc33-1de6446de5b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:54.860843 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:54.912674 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92cb7e4-678c-4d3a-bc33-1de6446de5b8 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:54.916669 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7c857-3de3-4cff-a695-5a9645208f43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:54.917423 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7c857-3de3-4cff-a695-5a9645208f43 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:54.920105 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7c857-3de3-4cff-a695-5a9645208f43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:54.920105 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef7c857-3de3-4cff-a695-5a9645208f43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:54.929194 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef7c857-3de3-4cff-a695-5a9645208f43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:54.930913 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef7c857-3de3-4cff-a695-5a9645208f43 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:54.942216 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1164/2324] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:34:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:34:55.096448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b2075-0784-4fef-b908-4c7355e6ad31 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:55.096448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-da4b2075-0784-4fef-b908-4c7355e6ad31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:55.098674 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:55.162556 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b2075-0784-4fef-b908-4c7355e6ad31 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:55.169067 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683a19b-c004-42e7-8e19-53364698495e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:55.169746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683a19b-c004-42e7-8e19-53364698495e service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:55.170078 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683a19b-c004-42e7-8e19-53364698495e service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:55.170403 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683a19b-c004-42e7-8e19-53364698495e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:55.172965 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b683a19b-c004-42e7-8e19-53364698495e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:55.264862 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b683a19b-c004-42e7-8e19-53364698495e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:55.266929 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b683a19b-c004-42e7-8e19-53364698495e service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:55.282355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1161/2325] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:34:55 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:34:55.447204 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7c857-3de3-4cff-a695-5a9645208f43 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:55.448714 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9ef7c857-3de3-4cff-a695-5a9645208f43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:55.462775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef7c857-3de3-4cff-a695-5a9645208f43 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:55.470745 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a535339d-0882-4c82-b9d5-fc66d616a013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:55.470745 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535339d-0882-4c82-b9d5-fc66d616a013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:55.470745 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535339d-0882-4c82-b9d5-fc66d616a013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:55.470745 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a535339d-0882-4c82-b9d5-fc66d616a013 admin admin] PATH_INFO: `/v3/projects/64574f0e703b42629d5948259c02dd84` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:55.471208 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a535339d-0882-4c82-b9d5-fc66d616a013 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64574f0e703b42629d5948259c02dd84)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:55.473396 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a535339d-0882-4c82-b9d5-fc66d616a013 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:55.611663 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a535339d-0882-4c82-b9d5-fc66d616a013 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64574f0e703b42629d5948259c02dd84', 'request_id': 'req-a535339d-0882-4c82-b9d5-fc66d616a013'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:55.696585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a535339d-0882-4c82-b9d5-fc66d616a013 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64574f0e703b42629d5948259c02dd84', 'request_id': 'req-a535339d-0882-4c82-b9d5-fc66d616a013'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:55.703137 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a535339d-0882-4c82-b9d5-fc66d616a013 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64574f0e703b42629d5948259c02dd84', 'request_id': 'req-a535339d-0882-4c82-b9d5-fc66d616a013'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:55.719898 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a535339d-0882-4c82-b9d5-fc66d616a013 admin admin] The token cache is being invalidate because project 64574f0e703b42629d5948259c02dd84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:34:55.720193 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a535339d-0882-4c82-b9d5-fc66d616a013 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:55.720854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a535339d-0882-4c82-b9d5-fc66d616a013 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:55.726046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1165/2326] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:34:55 2022] DELETE /identity/v3/projects/64574f0e703b42629d5948259c02dd84 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:55.727728 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb1340-5abe-44d4-a2bf-13cd28a2e4c7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:55.728190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb1340-5abe-44d4-a2bf-13cd28a2e4c7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:55.728350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdb1340-5abe-44d4-a2bf-13cd28a2e4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:55.750139 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ffdb1340-5abe-44d4-a2bf-13cd28a2e4c7 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:55.751382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ffdb1340-5abe-44d4-a2bf-13cd28a2e4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:55.815055 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1162/2327] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:55.823457 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d21f04-beda-4ca0-a377-0d015bf1030b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:55.823742 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d21f04-beda-4ca0-a377-0d015bf1030b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:55.824017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d21f04-beda-4ca0-a377-0d015bf1030b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:55.851014 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-74d21f04-beda-4ca0-a377-0d015bf1030b None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:55.851324 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb1340-5abe-44d4-a2bf-13cd28a2e4c7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:55.852344 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-74d21f04-beda-4ca0-a377-0d015bf1030b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:55.852717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ffdb1340-5abe-44d4-a2bf-13cd28a2e4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:55.866209 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1166/2328] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:55.866396 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdb1340-5abe-44d4-a2bf-13cd28a2e4c7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:55.869675 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d21f04-beda-4ca0-a377-0d015bf1030b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:55.876591 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98fd23-fc84-4b59-9b4b-c48adad3df13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:55.876591 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98fd23-fc84-4b59-9b4b-c48adad3df13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:55.876591 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98fd23-fc84-4b59-9b4b-c48adad3df13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:55.876591 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98fd23-fc84-4b59-9b4b-c48adad3df13 admin admin] PATH_INFO: `/v3/users/186a492ddee442b495cd8fce575f570c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:55.878026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba98fd23-fc84-4b59-9b4b-c48adad3df13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=186a492ddee442b495cd8fce575f570c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:55.879844 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d21f04-beda-4ca0-a377-0d015bf1030b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:55.880124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba98fd23-fc84-4b59-9b4b-c48adad3df13 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:55.885280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266f3bd-9e45-4b8b-8c80-af03ecd782fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:55.885914 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266f3bd-9e45-4b8b-8c80-af03ecd782fd admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:55.886237 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266f3bd-9e45-4b8b-8c80-af03ecd782fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:55.886532 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266f3bd-9e45-4b8b-8c80-af03ecd782fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:55.887228 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266f3bd-9e45-4b8b-8c80-af03ecd782fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:34:55.887538 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266f3bd-9e45-4b8b-8c80-af03ecd782fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:55.889157 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266f3bd-9e45-4b8b-8c80-af03ecd782fd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:55.901584 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1167/2329] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:34:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:34:55.906004 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266f3bd-9e45-4b8b-8c80-af03ecd782fd admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:55.917094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266f3bd-9e45-4b8b-8c80-af03ecd782fd admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:55.921895 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd2d029-d547-484e-a573-1efe0e55fef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:55.922503 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2d029-d547-484e-a573-1efe0e55fef0 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:55.922814 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2d029-d547-484e-a573-1efe0e55fef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:55.923091 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd2d029-d547-484e-a573-1efe0e55fef0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:55.923723 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd2d029-d547-484e-a573-1efe0e55fef0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:55.925739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd2d029-d547-484e-a573-1efe0e55fef0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:56.027263 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1168/2330] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:34:55 2022] POST /identity/v3/projects => generated 375 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:34:56.028395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd2d029-d547-484e-a573-1efe0e55fef0 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:56.078039 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ba98fd23-fc84-4b59-9b4b-c48adad3df13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '186a492ddee442b495cd8fce575f570c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:56.081220 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:56.101208 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:56.115602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd2d029-d547-484e-a573-1efe0e55fef0 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:56.122210 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b8faf-6e54-4f4d-a6f8-ef7b017b52c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:56.122210 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b8faf-6e54-4f4d-a6f8-ef7b017b52c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:56.122210 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b8faf-6e54-4f4d-a6f8-ef7b017b52c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:56.122618 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b8faf-6e54-4f4d-a6f8-ef7b017b52c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:56.122977 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b8faf-6e54-4f4d-a6f8-ef7b017b52c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:56.125149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b8faf-6e54-4f4d-a6f8-ef7b017b52c2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:56.153882 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ba98fd23-fc84-4b59-9b4b-c48adad3df13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '186a492ddee442b495cd8fce575f570c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:56.154131 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ba98fd23-fc84-4b59-9b4b-c48adad3df13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '186a492ddee442b495cd8fce575f570c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:56.154614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ba98fd23-fc84-4b59-9b4b-c48adad3df13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '186a492ddee442b495cd8fce575f570c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:56.168974 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ba98fd23-fc84-4b59-9b4b-c48adad3df13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '186a492ddee442b495cd8fce575f570c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:56.188619 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1163/2331] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:34:55 2022] DELETE /identity/v3/users/186a492ddee442b495cd8fce575f570c => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:34:56.192849 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ad8f5-dc95-4ed5-b675-b440d0d39432 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:56.193345 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ad8f5-dc95-4ed5-b675-b440d0d39432 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:56.193809 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ad8f5-dc95-4ed5-b675-b440d0d39432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:56.215699 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-5f3ad8f5-dc95-4ed5-b675-b440d0d39432 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:56.217007 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5f3ad8f5-dc95-4ed5-b675-b440d0d39432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:56.272116 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1169/2332] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:34:56 2022] POST /identity/v3/users => generated 288 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:34:56.275976 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b8faf-6e54-4f4d-a6f8-ef7b017b52c2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:56.277829 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-882b8faf-6e54-4f4d-a6f8-ef7b017b52c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:56.283718 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1164/2333] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:34:56.294422 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ad8f5-dc95-4ed5-b675-b440d0d39432 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:56.295433 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b8faf-6e54-4f4d-a6f8-ef7b017b52c2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:56.297658 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5f3ad8f5-dc95-4ed5-b675-b440d0d39432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:56.302491 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22825470-2208-4aa6-a79e-8199b2442c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:56.303151 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22825470-2208-4aa6-a79e-8199b2442c96 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:56.303430 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22825470-2208-4aa6-a79e-8199b2442c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:56.306403 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22825470-2208-4aa6-a79e-8199b2442c96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:56.306403 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22825470-2208-4aa6-a79e-8199b2442c96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:56.306730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22825470-2208-4aa6-a79e-8199b2442c96 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:56.317308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1170/2334] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:34:56.321080 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22825470-2208-4aa6-a79e-8199b2442c96 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:56.330940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22825470-2208-4aa6-a79e-8199b2442c96 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:56.335630 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88e9e4-c67e-4d0d-8bc6-e3529097e26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:56.336250 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88e9e4-c67e-4d0d-8bc6-e3529097e26f admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:56.336532 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88e9e4-c67e-4d0d-8bc6-e3529097e26f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:56.336907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88e9e4-c67e-4d0d-8bc6-e3529097e26f admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:56.337538 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88e9e4-c67e-4d0d-8bc6-e3529097e26f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:56.339172 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88e9e4-c67e-4d0d-8bc6-e3529097e26f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:56.350457 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1171/2335] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:34:56.355187 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88e9e4-c67e-4d0d-8bc6-e3529097e26f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:56.360369 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ad8f5-dc95-4ed5-b675-b440d0d39432 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:56.364153 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190de72-317c-4844-8306-8d1e2acf6eda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:56.364774 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190de72-317c-4844-8306-8d1e2acf6eda service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:56.365056 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190de72-317c-4844-8306-8d1e2acf6eda service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:56.365331 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190de72-317c-4844-8306-8d1e2acf6eda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:56.365435 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88e9e4-c67e-4d0d-8bc6-e3529097e26f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:56.371848 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5f233-0b9f-4011-a76c-58848e7b5f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:56.374358 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5f233-0b9f-4011-a76c-58848e7b5f85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:56.374358 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5f233-0b9f-4011-a76c-58848e7b5f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:56.374358 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5f233-0b9f-4011-a76c-58848e7b5f85 admin admin] PATH_INFO: `/v3/projects/f81d6a5715cf497fbda2b68647c0a5a2/users/d304395e2a0a49ceae4c4d9166ca92e2/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:56.377680 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b190de72-317c-4844-8306-8d1e2acf6eda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:56.379161 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b190de72-317c-4844-8306-8d1e2acf6eda service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:56.389211 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1165/2336] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:34:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:34:56.391148 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da5f233-0b9f-4011-a76c-58848e7b5f85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f81d6a5715cf497fbda2b68647c0a5a2, user_id=d304395e2a0a49ceae4c4d9166ca92e2, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:56.394270 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da5f233-0b9f-4011-a76c-58848e7b5f85 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:56.473412 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1172/2337] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:34:56 2022] PUT /identity/v3/projects/f81d6a5715cf497fbda2b68647c0a5a2/users/d304395e2a0a49ceae4c4d9166ca92e2/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:56.484996 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f5e7d-cf27-4419-bf51-4dd23037fff1 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:56.485287 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f5e7d-cf27-4419-bf51-4dd23037fff1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:56.485563 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f5e7d-cf27-4419-bf51-4dd23037fff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:56.517453 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7c5f5e7d-cf27-4419-bf51-4dd23037fff1 None None] MFA Rules not processed for user `d304395e2a0a49ceae4c4d9166ca92e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:56.518670 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7c5f5e7d-cf27-4419-bf51-4dd23037fff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:56.580336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:56.586624 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1166/2338] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:56.599714 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5f233-0b9f-4011-a76c-58848e7b5f85 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:56.604245 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2da5f233-0b9f-4011-a76c-58848e7b5f85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:56.669004 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5f233-0b9f-4011-a76c-58848e7b5f85 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:56.672609 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28269797-3f71-4912-a3c7-d14f0e49d391 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:56.673217 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28269797-3f71-4912-a3c7-d14f0e49d391 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:56.673526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28269797-3f71-4912-a3c7-d14f0e49d391 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:56.673794 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28269797-3f71-4912-a3c7-d14f0e49d391 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:56.687816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28269797-3f71-4912-a3c7-d14f0e49d391 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:56.690117 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28269797-3f71-4912-a3c7-d14f0e49d391 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:56.700211 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1173/2339] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:34:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:34:56.823551 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f5e7d-cf27-4419-bf51-4dd23037fff1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:56.825045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7c5f5e7d-cf27-4419-bf51-4dd23037fff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:56.901872 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f5e7d-cf27-4419-bf51-4dd23037fff1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:56.906208 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e19d0c-051a-448e-b327-0921af0008ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:56.906858 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e19d0c-051a-448e-b327-0921af0008ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:56.907140 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e19d0c-051a-448e-b327-0921af0008ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:56.907440 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e19d0c-051a-448e-b327-0921af0008ee admin admin] PATH_INFO: `/v3/projects/d7ec12d875ef449eb2b95bd275c4c6b0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:56.908583 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e19d0c-051a-448e-b327-0921af0008ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7ec12d875ef449eb2b95bd275c4c6b0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:56.910482 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e19d0c-051a-448e-b327-0921af0008ee admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:57.034804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f8e19d0c-051a-448e-b327-0921af0008ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7ec12d875ef449eb2b95bd275c4c6b0', 'request_id': 'req-f8e19d0c-051a-448e-b327-0921af0008ee'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:57.040368 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f8e19d0c-051a-448e-b327-0921af0008ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7ec12d875ef449eb2b95bd275c4c6b0', 'request_id': 'req-f8e19d0c-051a-448e-b327-0921af0008ee'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:57.046442 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f8e19d0c-051a-448e-b327-0921af0008ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7ec12d875ef449eb2b95bd275c4c6b0', 'request_id': 'req-f8e19d0c-051a-448e-b327-0921af0008ee'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:57.146931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f8e19d0c-051a-448e-b327-0921af0008ee admin admin] The token cache is being invalidate because project d7ec12d875ef449eb2b95bd275c4c6b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:34:57.153032 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f8e19d0c-051a-448e-b327-0921af0008ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:57.153032 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-f8e19d0c-051a-448e-b327-0921af0008ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:57.153032 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1167/2340] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:34:56 2022] DELETE /identity/v3/projects/d7ec12d875ef449eb2b95bd275c4c6b0 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:57.154372 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28269797-3f71-4912-a3c7-d14f0e49d391 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:57.165025 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28269797-3f71-4912-a3c7-d14f0e49d391 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:57.228291 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ff861-c88d-4b0a-88ce-862f8b0b4d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:57.229520 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ff861-c88d-4b0a-88ce-862f8b0b4d73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:57.229868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ff861-c88d-4b0a-88ce-862f8b0b4d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:57.233517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ff861-c88d-4b0a-88ce-862f8b0b4d73 admin admin] PATH_INFO: `/v3/users/3ca05db84f454d46a741e4414ef3586d` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:57.233517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ff861-c88d-4b0a-88ce-862f8b0b4d73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ca05db84f454d46a741e4414ef3586d)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:57.233517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ff861-c88d-4b0a-88ce-862f8b0b4d73 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:57.495176 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:57.501150 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-930ff861-c88d-4b0a-88ce-862f8b0b4d73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ca05db84f454d46a741e4414ef3586d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:57.510461 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-930ff861-c88d-4b0a-88ce-862f8b0b4d73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ca05db84f454d46a741e4414ef3586d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:57.604226 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-930ff861-c88d-4b0a-88ce-862f8b0b4d73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ca05db84f454d46a741e4414ef3586d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:57.604871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-930ff861-c88d-4b0a-88ce-862f8b0b4d73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ca05db84f454d46a741e4414ef3586d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:57.617602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-930ff861-c88d-4b0a-88ce-862f8b0b4d73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ca05db84f454d46a741e4414ef3586d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:57.627334 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1174/2341] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:34:57 2022] DELETE /identity/v3/users/3ca05db84f454d46a741e4414ef3586d => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:58.046813 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e19d0c-051a-448e-b327-0921af0008ee admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:58.048393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f8e19d0c-051a-448e-b327-0921af0008ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:58.068336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e19d0c-051a-448e-b327-0921af0008ee admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:58.143108 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7d152-35a9-4064-a3d1-2532721536ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:58.143797 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7d152-35a9-4064-a3d1-2532721536ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:58.144076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7d152-35a9-4064-a3d1-2532721536ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:58.144342 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7d152-35a9-4064-a3d1-2532721536ee admin admin] PATH_INFO: `/v3/projects/b964110459f54020ae3963e8aba796f4` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:58.145853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f7d152-35a9-4064-a3d1-2532721536ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=b964110459f54020ae3963e8aba796f4)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:58.148797 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f7d152-35a9-4064-a3d1-2532721536ee admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:58.285492 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:58.299963 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-55f7d152-35a9-4064-a3d1-2532721536ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b964110459f54020ae3963e8aba796f4', 'request_id': 'req-55f7d152-35a9-4064-a3d1-2532721536ee'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:58.305472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-55f7d152-35a9-4064-a3d1-2532721536ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b964110459f54020ae3963e8aba796f4', 'request_id': 'req-55f7d152-35a9-4064-a3d1-2532721536ee'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:58.312412 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-55f7d152-35a9-4064-a3d1-2532721536ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b964110459f54020ae3963e8aba796f4', 'request_id': 'req-55f7d152-35a9-4064-a3d1-2532721536ee'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:58.406948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-55f7d152-35a9-4064-a3d1-2532721536ee admin admin] The token cache is being invalidate because project b964110459f54020ae3963e8aba796f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:34:58.407332 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-55f7d152-35a9-4064-a3d1-2532721536ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:58.408061 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-55f7d152-35a9-4064-a3d1-2532721536ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:34:58.410023 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1168/2342] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:34:58 2022] DELETE /identity/v3/projects/b964110459f54020ae3963e8aba796f4 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:58.423149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc323a4d-6b6a-4a43-abdd-233ca39d20e2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:58.423699 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc323a4d-6b6a-4a43-abdd-233ca39d20e2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:58.424230 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc323a4d-6b6a-4a43-abdd-233ca39d20e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:58.448274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-dc323a4d-6b6a-4a43-abdd-233ca39d20e2 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:58.449565 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-dc323a4d-6b6a-4a43-abdd-233ca39d20e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:58.519999 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1175/2343] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:58.530960 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64744eab-7700-44a9-9b42-bb5dcb0a4d48 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:58.531249 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64744eab-7700-44a9-9b42-bb5dcb0a4d48 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:58.531754 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64744eab-7700-44a9-9b42-bb5dcb0a4d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:58.552806 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-64744eab-7700-44a9-9b42-bb5dcb0a4d48 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:58.553962 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-64744eab-7700-44a9-9b42-bb5dcb0a4d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:58.562454 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1169/2344] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:58.566895 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc323a4d-6b6a-4a43-abdd-233ca39d20e2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:58.577997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc323a4d-6b6a-4a43-abdd-233ca39d20e2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:58.582729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67047cee-0e34-4ea4-babf-25fe4c1c1521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:58.583370 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67047cee-0e34-4ea4-babf-25fe4c1c1521 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:58.583646 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67047cee-0e34-4ea4-babf-25fe4c1c1521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:58.583913 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67047cee-0e34-4ea4-babf-25fe4c1c1521 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:58.584557 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67047cee-0e34-4ea4-babf-25fe4c1c1521 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:34:58.584855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67047cee-0e34-4ea4-babf-25fe4c1c1521 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:58.587292 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67047cee-0e34-4ea4-babf-25fe4c1c1521 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:58.599612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1176/2345] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:34:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:34:58.608445 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64744eab-7700-44a9-9b42-bb5dcb0a4d48 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:58.632812 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64744eab-7700-44a9-9b42-bb5dcb0a4d48 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:58.632812 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cac24d-aa4d-450f-a75c-06a6dba6630d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:58.632812 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cac24d-aa4d-450f-a75c-06a6dba6630d admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:58.632812 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cac24d-aa4d-450f-a75c-06a6dba6630d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:58.633160 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cac24d-aa4d-450f-a75c-06a6dba6630d admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:58.633160 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cac24d-aa4d-450f-a75c-06a6dba6630d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:58.633160 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cac24d-aa4d-450f-a75c-06a6dba6630d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:58.712789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1170/2346] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:34:58 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:34:58.713635 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67047cee-0e34-4ea4-babf-25fe4c1c1521 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:58.724011 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67047cee-0e34-4ea4-babf-25fe4c1c1521 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:58.781370 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874567a4-391b-4e5c-88a0-0385cdb2046c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:58.781979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874567a4-391b-4e5c-88a0-0385cdb2046c admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:58.782255 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874567a4-391b-4e5c-88a0-0385cdb2046c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:58.782526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874567a4-391b-4e5c-88a0-0385cdb2046c admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:58.783189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874567a4-391b-4e5c-88a0-0385cdb2046c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:58.786596 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874567a4-391b-4e5c-88a0-0385cdb2046c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:58.862634 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1177/2347] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:34:58 2022] POST /identity/v3/users => generated 288 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:34:58.866160 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cac24d-aa4d-450f-a75c-06a6dba6630d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:58.882796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cac24d-aa4d-450f-a75c-06a6dba6630d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:58.888348 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777dc0d3-3fc1-4c4f-a4c2-53e06cc351d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:58.888966 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777dc0d3-3fc1-4c4f-a4c2-53e06cc351d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:58.889234 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777dc0d3-3fc1-4c4f-a4c2-53e06cc351d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:58.889508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777dc0d3-3fc1-4c4f-a4c2-53e06cc351d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:58.890183 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777dc0d3-3fc1-4c4f-a4c2-53e06cc351d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:58.891811 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777dc0d3-3fc1-4c4f-a4c2-53e06cc351d4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:58.901979 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1171/2348] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:34:58.906304 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874567a4-391b-4e5c-88a0-0385cdb2046c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:58.916476 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874567a4-391b-4e5c-88a0-0385cdb2046c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:58.921117 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258fc56-d5a1-4d50-9bf6-3203a433b825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:58.924254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258fc56-d5a1-4d50-9bf6-3203a433b825 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:58.924254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258fc56-d5a1-4d50-9bf6-3203a433b825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:58.924254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258fc56-d5a1-4d50-9bf6-3203a433b825 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:58.924254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e258fc56-d5a1-4d50-9bf6-3203a433b825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:58.924660 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e258fc56-d5a1-4d50-9bf6-3203a433b825 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:58.935719 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1178/2349] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:34:58.940166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777dc0d3-3fc1-4c4f-a4c2-53e06cc351d4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:58.950590 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777dc0d3-3fc1-4c4f-a4c2-53e06cc351d4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:58.955212 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49465e0c-1c4f-4848-8ee1-5fe30dbeace4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:58.955849 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49465e0c-1c4f-4848-8ee1-5fe30dbeace4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:58.956130 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49465e0c-1c4f-4848-8ee1-5fe30dbeace4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:58.956409 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49465e0c-1c4f-4848-8ee1-5fe30dbeace4 admin admin] PATH_INFO: `/v3/projects/a74a8dc588bf4dfe9fb9165ffe948e8c/users/3e103dd2c68242499a8c7eae41b9d183/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:58.971220 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49465e0c-1c4f-4848-8ee1-5fe30dbeace4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a74a8dc588bf4dfe9fb9165ffe948e8c, user_id=3e103dd2c68242499a8c7eae41b9d183, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:58.973858 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49465e0c-1c4f-4848-8ee1-5fe30dbeace4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:59.026723 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1172/2350] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:34:58 2022] PUT /identity/v3/projects/a74a8dc588bf4dfe9fb9165ffe948e8c/users/3e103dd2c68242499a8c7eae41b9d183/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:59.038108 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35780b3a-17c9-41b0-95d5-71f9120c7ba0 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:59.038469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35780b3a-17c9-41b0-95d5-71f9120c7ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:59.038770 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35780b3a-17c9-41b0-95d5-71f9120c7ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:59.078216 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-35780b3a-17c9-41b0-95d5-71f9120c7ba0 None None] MFA Rules not processed for user `3e103dd2c68242499a8c7eae41b9d183`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:59.079630 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-35780b3a-17c9-41b0-95d5-71f9120c7ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:59.091714 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:59.149246 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1179/2351] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:59.153857 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49465e0c-1c4f-4848-8ee1-5fe30dbeace4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:59.220211 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49465e0c-1c4f-4848-8ee1-5fe30dbeace4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:59.225190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38cef4-fd90-4e4a-8d09-955f704dc008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:59.225782 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38cef4-fd90-4e4a-8d09-955f704dc008 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:59.226088 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38cef4-fd90-4e4a-8d09-955f704dc008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:59.226361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38cef4-fd90-4e4a-8d09-955f704dc008 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:59.227535 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c38cef4-fd90-4e4a-8d09-955f704dc008 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:59.229456 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c38cef4-fd90-4e4a-8d09-955f704dc008 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:59.352897 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1173/2352] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:34:59 2022] POST /identity/v3/projects => generated 377 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:34:59.353943 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35780b3a-17c9-41b0-95d5-71f9120c7ba0 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:59.364981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35780b3a-17c9-41b0-95d5-71f9120c7ba0 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:59.426689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806f089-e6ea-445d-af35-ae6c8dcf5ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:59.427369 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806f089-e6ea-445d-af35-ae6c8dcf5ac7 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:59.427647 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806f089-e6ea-445d-af35-ae6c8dcf5ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:59.427919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d806f089-e6ea-445d-af35-ae6c8dcf5ac7 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:59.428643 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806f089-e6ea-445d-af35-ae6c8dcf5ac7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:59.431222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d806f089-e6ea-445d-af35-ae6c8dcf5ac7 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:59.526068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1180/2353] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:34:59 2022] POST /identity/v3/users => generated 295 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:34:59.531571 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38cef4-fd90-4e4a-8d09-955f704dc008 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:59.543178 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38cef4-fd90-4e4a-8d09-955f704dc008 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:59.548185 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a665f0-4050-48ad-aacd-792e788fb495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:59.548827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a665f0-4050-48ad-aacd-792e788fb495 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:59.549113 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a665f0-4050-48ad-aacd-792e788fb495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:59.549387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a665f0-4050-48ad-aacd-792e788fb495 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:59.550186 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a665f0-4050-48ad-aacd-792e788fb495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:59.551997 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a665f0-4050-48ad-aacd-792e788fb495 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:59.564576 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1174/2354] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:34:59.568618 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806f089-e6ea-445d-af35-ae6c8dcf5ac7 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:59.580367 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d806f089-e6ea-445d-af35-ae6c8dcf5ac7 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:59.585242 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3edfb9-9d07-418e-954f-fa9f10e7112c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:59.585532 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3edfb9-9d07-418e-954f-fa9f10e7112c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:59.585806 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3edfb9-9d07-418e-954f-fa9f10e7112c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:59.591104 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd92b1-96cd-4a41-92bb-4c0ca54ac976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:59.591776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd92b1-96cd-4a41-92bb-4c0ca54ac976 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:59.592056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd92b1-96cd-4a41-92bb-4c0ca54ac976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:59.592339 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd92b1-96cd-4a41-92bb-4c0ca54ac976 admin admin] PATH_INFO: `/v3/projects/d553fa0200ef453e846ab02f3a3cdd3e/users/7eb17135812e48c68c11eb0d82e50e43/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:59.605944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6d3edfb9-9d07-418e-954f-fa9f10e7112c None None] MFA Rules not processed for user `d304395e2a0a49ceae4c4d9166ca92e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:59.607452 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6d3edfb9-9d07-418e-954f-fa9f10e7112c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:59.608154 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd92b1-96cd-4a41-92bb-4c0ca54ac976 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d553fa0200ef453e846ab02f3a3cdd3e, user_id=7eb17135812e48c68c11eb0d82e50e43, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:59.610950 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd92b1-96cd-4a41-92bb-4c0ca54ac976 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:59.635890 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:34:59.666377 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1175/2355] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:59.678748 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1181/2356] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:34:59 2022] PUT /identity/v3/projects/d553fa0200ef453e846ab02f3a3cdd3e/users/7eb17135812e48c68c11eb0d82e50e43/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:59.683281 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3edfb9-9d07-418e-954f-fa9f10e7112c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:59.694083 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3edfb9-9d07-418e-954f-fa9f10e7112c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:59.762388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2fcd9-1200-452e-be07-da09a4accc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:59.763286 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2fcd9-1200-452e-be07-da09a4accc72 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:59.763547 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2fcd9-1200-452e-be07-da09a4accc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:59.763808 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2fcd9-1200-452e-be07-da09a4accc72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:59.764698 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d2fcd9-1200-452e-be07-da09a4accc72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:59.766666 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d2fcd9-1200-452e-be07-da09a4accc72 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:59.780665 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1176/2357] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:34:59 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:34:59.785004 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd92b1-96cd-4a41-92bb-4c0ca54ac976 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:59.795831 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd92b1-96cd-4a41-92bb-4c0ca54ac976 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:34:59.800479 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8666c2-580e-4f3e-8ce1-1f729067dad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:34:59.801123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8666c2-580e-4f3e-8ce1-1f729067dad9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:59.801425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8666c2-580e-4f3e-8ce1-1f729067dad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:59.801696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8666c2-580e-4f3e-8ce1-1f729067dad9 admin admin] PATH_INFO: `/v3/projects/d553fa0200ef453e846ab02f3a3cdd3e/users/7eb17135812e48c68c11eb0d82e50e43/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:59.803831 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8666c2-580e-4f3e-8ce1-1f729067dad9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d553fa0200ef453e846ab02f3a3cdd3e, user_id=7eb17135812e48c68c11eb0d82e50e43, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:34:59.808428 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8666c2-580e-4f3e-8ce1-1f729067dad9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:34:59.819832 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1182/2358] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:34:59 2022] PUT /identity/v3/projects/d553fa0200ef453e846ab02f3a3cdd3e/users/7eb17135812e48c68c11eb0d82e50e43/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:34:59.830842 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7c172-8b11-4390-9d78-e52567836d5f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:34:59.831123 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7c172-8b11-4390-9d78-e52567836d5f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:34:59.831450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7c172-8b11-4390-9d78-e52567836d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:34:59.865683 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-51a7c172-8b11-4390-9d78-e52567836d5f None None] MFA Rules not processed for user `7eb17135812e48c68c11eb0d82e50e43`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:34:59.867208 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-51a7c172-8b11-4390-9d78-e52567836d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:34:59.947877 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1177/2359] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:34:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:34:59.970048 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8666c2-580e-4f3e-8ce1-1f729067dad9 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:34:59.972336 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bd8666c2-580e-4f3e-8ce1-1f729067dad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:35:00.044461 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8666c2-580e-4f3e-8ce1-1f729067dad9 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:35:00.048178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123703c-e9ce-4e3a-b34b-1ed09e117425 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:35:00.048851 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123703c-e9ce-4e3a-b34b-1ed09e117425 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:35:00.049195 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123703c-e9ce-4e3a-b34b-1ed09e117425 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:35:00.049523 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7123703c-e9ce-4e3a-b34b-1ed09e117425 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:35:00.069535 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:35:00.121914 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7123703c-e9ce-4e3a-b34b-1ed09e117425 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:35:00.123583 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7123703c-e9ce-4e3a-b34b-1ed09e117425 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:35:00.132329 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1183/2360] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:34:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:35:02.960241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfb1ea-ff72-418f-aa94-ca1f02df9b20 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:35:02.960702 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfb1ea-ff72-418f-aa94-ca1f02df9b20 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:35:02.960821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cfb1ea-ff72-418f-aa94-ca1f02df9b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:35:02.979732 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-86cfb1ea-ff72-418f-aa94-ca1f02df9b20 None None] MFA Rules not processed for user `c79aaf1f72634eefb5e89c8005afe54b`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:35:02.980924 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-86cfb1ea-ff72-418f-aa94-ca1f02df9b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:35:03.033974 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1178/2361] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:35:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:35:10.164652 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=18 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:35:13.027494 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:35:14.326580 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123703c-e9ce-4e3a-b34b-1ed09e117425 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:35:14.328048 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7123703c-e9ce-4e3a-b34b-1ed09e117425 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:35:14.433010 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7123703c-e9ce-4e3a-b34b-1ed09e117425 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:35:14.437554 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb222151-a0f0-4661-8a75-857e8ba48988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:35:14.438217 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb222151-a0f0-4661-8a75-857e8ba48988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:35:14.438495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb222151-a0f0-4661-8a75-857e8ba48988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:35:14.438805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb222151-a0f0-4661-8a75-857e8ba48988 admin admin] PATH_INFO: `/v3/users/72ab7ad1bf5d484494b087365e410dca` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:35:14.439974 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb222151-a0f0-4661-8a75-857e8ba48988 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72ab7ad1bf5d484494b087365e410dca)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:35:14.441903 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb222151-a0f0-4661-8a75-857e8ba48988 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:35:14.627880 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fb222151-a0f0-4661-8a75-857e8ba48988 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72ab7ad1bf5d484494b087365e410dca'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:35:14.634666 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fb222151-a0f0-4661-8a75-857e8ba48988 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72ab7ad1bf5d484494b087365e410dca'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:35:14.683134 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fb222151-a0f0-4661-8a75-857e8ba48988 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72ab7ad1bf5d484494b087365e410dca'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:35:14.683755 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fb222151-a0f0-4661-8a75-857e8ba48988 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72ab7ad1bf5d484494b087365e410dca'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:35:14.695849 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fb222151-a0f0-4661-8a75-857e8ba48988 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72ab7ad1bf5d484494b087365e410dca'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:35:14.701765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1184/2362] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:35:14 2022] DELETE /identity/v3/users/72ab7ad1bf5d484494b087365e410dca => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:35:14.706316 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84694d3-d413-4371-b95c-2ba64b220c57 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:35:14.706822 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84694d3-d413-4371-b95c-2ba64b220c57 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:35:14.706919 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84694d3-d413-4371-b95c-2ba64b220c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:35:14.725940 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e84694d3-d413-4371-b95c-2ba64b220c57 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:35:14.727795 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e84694d3-d413-4371-b95c-2ba64b220c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:35:14.785789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1179/2363] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:35:14.795882 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb222151-a0f0-4661-8a75-857e8ba48988 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:35:14.797301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fb222151-a0f0-4661-8a75-857e8ba48988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:35:14.853394 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb222151-a0f0-4661-8a75-857e8ba48988 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:35:14.857202 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52fe84d-4617-49ac-98b4-d0002e75461b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:35:14.857814 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52fe84d-4617-49ac-98b4-d0002e75461b service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:35:14.858092 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52fe84d-4617-49ac-98b4-d0002e75461b service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:35:14.858362 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52fe84d-4617-49ac-98b4-d0002e75461b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:35:14.867478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52fe84d-4617-49ac-98b4-d0002e75461b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:35:14.868895 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52fe84d-4617-49ac-98b4-d0002e75461b service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:35:14.878678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1185/2364] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:35:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:35:15.404344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84694d3-d413-4371-b95c-2ba64b220c57 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:35:15.405780 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e84694d3-d413-4371-b95c-2ba64b220c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:35:15.420451 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84694d3-d413-4371-b95c-2ba64b220c57 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:35:15.427885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68a46b-67e1-43b4-a262-43cdde076bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:35:15.427885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68a46b-67e1-43b4-a262-43cdde076bd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:35:15.427885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68a46b-67e1-43b4-a262-43cdde076bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:35:15.427885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68a46b-67e1-43b4-a262-43cdde076bd3 admin admin] PATH_INFO: `/v3/projects/e909518e56ca454b8fbd120daa8e7e8e` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:35:15.429116 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68a46b-67e1-43b4-a262-43cdde076bd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e909518e56ca454b8fbd120daa8e7e8e)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:35:15.431434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68a46b-67e1-43b4-a262-43cdde076bd3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:35:15.538111 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fa68a46b-67e1-43b4-a262-43cdde076bd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e909518e56ca454b8fbd120daa8e7e8e', 'request_id': 'req-fa68a46b-67e1-43b4-a262-43cdde076bd3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:35:15.548162 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fa68a46b-67e1-43b4-a262-43cdde076bd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e909518e56ca454b8fbd120daa8e7e8e', 'request_id': 'req-fa68a46b-67e1-43b4-a262-43cdde076bd3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:35:15.555968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fa68a46b-67e1-43b4-a262-43cdde076bd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e909518e56ca454b8fbd120daa8e7e8e', 'request_id': 'req-fa68a46b-67e1-43b4-a262-43cdde076bd3'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:35:15.626107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fa68a46b-67e1-43b4-a262-43cdde076bd3 admin admin] The token cache is being invalidate because project e909518e56ca454b8fbd120daa8e7e8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:35:15.626400 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fa68a46b-67e1-43b4-a262-43cdde076bd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:35:15.627141 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-fa68a46b-67e1-43b4-a262-43cdde076bd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:35:15.628759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1180/2365] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:35:15 2022] DELETE /identity/v3/projects/e909518e56ca454b8fbd120daa8e7e8e => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:35:15.632719 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52fe84d-4617-49ac-98b4-d0002e75461b service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:35:15.644651 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52fe84d-4617-49ac-98b4-d0002e75461b service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:35:15.663231 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:35:15.700566 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a70fc3-3962-4017-81e6-595669016fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:35:15.701185 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a70fc3-3962-4017-81e6-595669016fb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:35:15.701869 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a70fc3-3962-4017-81e6-595669016fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:35:15.702128 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a70fc3-3962-4017-81e6-595669016fb9 admin admin] PATH_INFO: `/v3/users/76d7a675bcfd4c21bc6da196f21647f1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:35:15.703271 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a70fc3-3962-4017-81e6-595669016fb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76d7a675bcfd4c21bc6da196f21647f1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:35:15.705172 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a70fc3-3962-4017-81e6-595669016fb9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:35:15.878655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-06a70fc3-3962-4017-81e6-595669016fb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76d7a675bcfd4c21bc6da196f21647f1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:35:15.885534 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-06a70fc3-3962-4017-81e6-595669016fb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76d7a675bcfd4c21bc6da196f21647f1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:35:15.957331 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-06a70fc3-3962-4017-81e6-595669016fb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76d7a675bcfd4c21bc6da196f21647f1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:35:15.957902 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-06a70fc3-3962-4017-81e6-595669016fb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76d7a675bcfd4c21bc6da196f21647f1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:35:15.970026 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-06a70fc3-3962-4017-81e6-595669016fb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76d7a675bcfd4c21bc6da196f21647f1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:35:15.975826 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1186/2366] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:35:15 2022] DELETE /identity/v3/users/76d7a675bcfd4c21bc6da196f21647f1 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:35:16.399633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68a46b-67e1-43b4-a262-43cdde076bd3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:35:16.401180 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fa68a46b-67e1-43b4-a262-43cdde076bd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:35:16.417331 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68a46b-67e1-43b4-a262-43cdde076bd3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:35:16.474358 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc09466-e27a-42f5-a8e7-eb12c59aaf08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:35:16.475028 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc09466-e27a-42f5-a8e7-eb12c59aaf08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:35:16.475305 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc09466-e27a-42f5-a8e7-eb12c59aaf08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:35:16.475579 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc09466-e27a-42f5-a8e7-eb12c59aaf08 admin admin] PATH_INFO: `/v3/projects/4ec9aade863042379abbdfa76c30b1b7` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:35:16.476826 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc09466-e27a-42f5-a8e7-eb12c59aaf08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ec9aade863042379abbdfa76c30b1b7)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:35:16.478789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc09466-e27a-42f5-a8e7-eb12c59aaf08 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:35:16.571035 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:35:16.587845 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1cc09466-e27a-42f5-a8e7-eb12c59aaf08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ec9aade863042379abbdfa76c30b1b7', 'request_id': 'req-1cc09466-e27a-42f5-a8e7-eb12c59aaf08'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:35:16.593164 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1cc09466-e27a-42f5-a8e7-eb12c59aaf08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ec9aade863042379abbdfa76c30b1b7', 'request_id': 'req-1cc09466-e27a-42f5-a8e7-eb12c59aaf08'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:35:16.599362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1cc09466-e27a-42f5-a8e7-eb12c59aaf08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ec9aade863042379abbdfa76c30b1b7', 'request_id': 'req-1cc09466-e27a-42f5-a8e7-eb12c59aaf08'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:35:16.695030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1cc09466-e27a-42f5-a8e7-eb12c59aaf08 admin admin] The token cache is being invalidate because project 4ec9aade863042379abbdfa76c30b1b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:35:16.695864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1cc09466-e27a-42f5-a8e7-eb12c59aaf08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:35:16.696764 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1cc09466-e27a-42f5-a8e7-eb12c59aaf08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:35:16.701913 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1181/2367] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:35:16 2022] DELETE /identity/v3/projects/4ec9aade863042379abbdfa76c30b1b7 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:35:16.705631 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016fedac-39c4-42a7-a72f-9e151e4d9e36 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:35:16.706127 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016fedac-39c4-42a7-a72f-9e151e4d9e36 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:35:16.706243 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016fedac-39c4-42a7-a72f-9e151e4d9e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:35:16.730778 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-016fedac-39c4-42a7-a72f-9e151e4d9e36 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:35:16.730778 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-016fedac-39c4-42a7-a72f-9e151e4d9e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:35:16.807438 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1187/2368] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:35:16.816903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc23c8d-d69a-4b66-aa4c-2195fca881df None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:35:16.817281 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc23c8d-d69a-4b66-aa4c-2195fca881df None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:35:16.817564 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc23c8d-d69a-4b66-aa4c-2195fca881df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:35:16.837698 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-dcc23c8d-d69a-4b66-aa4c-2195fca881df None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:35:16.838932 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-dcc23c8d-d69a-4b66-aa4c-2195fca881df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:35:16.846785 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1182/2369] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:35:16.851071 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016fedac-39c4-42a7-a72f-9e151e4d9e36 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:35:16.860927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016fedac-39c4-42a7-a72f-9e151e4d9e36 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:35:16.865481 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc299e00-24fb-4e9f-b290-cd052ea97ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:35:16.866077 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc299e00-24fb-4e9f-b290-cd052ea97ead admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:35:16.866355 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc299e00-24fb-4e9f-b290-cd052ea97ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:35:16.866626 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc299e00-24fb-4e9f-b290-cd052ea97ead admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:35:16.867889 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc299e00-24fb-4e9f-b290-cd052ea97ead admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:35:16.868190 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc299e00-24fb-4e9f-b290-cd052ea97ead admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:35:16.869774 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc299e00-24fb-4e9f-b290-cd052ea97ead admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:35:16.881868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1188/2370] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:35:16.887058 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc23c8d-d69a-4b66-aa4c-2195fca881df None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:35:16.896148 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc23c8d-d69a-4b66-aa4c-2195fca881df None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:35:16.900829 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c85177a-effc-4d5c-9686-9557434e632d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:35:16.901417 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85177a-effc-4d5c-9686-9557434e632d admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:35:16.901695 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85177a-effc-4d5c-9686-9557434e632d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:35:16.901965 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c85177a-effc-4d5c-9686-9557434e632d admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:35:16.902584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c85177a-effc-4d5c-9686-9557434e632d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:35:16.904491 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c85177a-effc-4d5c-9686-9557434e632d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:35:17.027361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1183/2371] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:35:16 2022] POST /identity/v3/projects => generated 377 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:35:17.031330 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc299e00-24fb-4e9f-b290-cd052ea97ead admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:35:17.041169 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc299e00-24fb-4e9f-b290-cd052ea97ead admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:35:17.067174 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:35:17.096951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2542ea0-e21b-40b5-ae09-0d09a4a25409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:35:17.097547 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2542ea0-e21b-40b5-ae09-0d09a4a25409 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:35:17.097872 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2542ea0-e21b-40b5-ae09-0d09a4a25409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:35:17.098109 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2542ea0-e21b-40b5-ae09-0d09a4a25409 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:35:17.098747 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2542ea0-e21b-40b5-ae09-0d09a4a25409 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:35:17.100658 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2542ea0-e21b-40b5-ae09-0d09a4a25409 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:35:17.180806 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1189/2372] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:35:17 2022] POST /identity/v3/users => generated 289 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:35:17.185004 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c85177a-effc-4d5c-9686-9557434e632d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:35:17.196245 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c85177a-effc-4d5c-9686-9557434e632d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:35:17.201204 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be51db0-e7a0-4ad4-a911-2d6abe83e9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:35:17.201775 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be51db0-e7a0-4ad4-a911-2d6abe83e9f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:35:17.202057 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be51db0-e7a0-4ad4-a911-2d6abe83e9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:35:17.202369 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be51db0-e7a0-4ad4-a911-2d6abe83e9f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:35:17.203029 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be51db0-e7a0-4ad4-a911-2d6abe83e9f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:35:17.204625 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be51db0-e7a0-4ad4-a911-2d6abe83e9f4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:35:17.215139 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1184/2373] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:35:17.221297 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2542ea0-e21b-40b5-ae09-0d09a4a25409 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:35:17.232023 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2542ea0-e21b-40b5-ae09-0d09a4a25409 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:35:17.236836 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6338a-c805-4e00-87a5-18fc59294e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:35:17.237431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6338a-c805-4e00-87a5-18fc59294e5a admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:35:17.237716 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6338a-c805-4e00-87a5-18fc59294e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:35:17.237993 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6338a-c805-4e00-87a5-18fc59294e5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:35:17.238649 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe6338a-c805-4e00-87a5-18fc59294e5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:35:17.241425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe6338a-c805-4e00-87a5-18fc59294e5a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:35:17.252145 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1190/2374] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:35:17.256327 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be51db0-e7a0-4ad4-a911-2d6abe83e9f4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:35:17.266212 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be51db0-e7a0-4ad4-a911-2d6abe83e9f4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:35:17.270815 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56d29f-1017-4170-a050-ee22169434a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:35:17.271459 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56d29f-1017-4170-a050-ee22169434a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:35:17.271738 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56d29f-1017-4170-a050-ee22169434a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:35:17.272011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d56d29f-1017-4170-a050-ee22169434a5 admin admin] PATH_INFO: `/v3/projects/82e9167182ee42e18d58cdd818374c22/users/d2c6655886ed4cde8617b1b4d50be464/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:35:17.286126 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d56d29f-1017-4170-a050-ee22169434a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82e9167182ee42e18d58cdd818374c22, user_id=d2c6655886ed4cde8617b1b4d50be464, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:35:17.288710 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d56d29f-1017-4170-a050-ee22169434a5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:35:17.357467 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1185/2375] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:35:17 2022] PUT /identity/v3/projects/82e9167182ee42e18d58cdd818374c22/users/d2c6655886ed4cde8617b1b4d50be464/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:35:17.369211 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a307cda-1bef-4a68-87a8-d5156d230c9b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:35:17.369495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a307cda-1bef-4a68-87a8-d5156d230c9b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:35:17.369768 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a307cda-1bef-4a68-87a8-d5156d230c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:35:17.398290 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0a307cda-1bef-4a68-87a8-d5156d230c9b None None] MFA Rules not processed for user `d2c6655886ed4cde8617b1b4d50be464`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:35:17.399607 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0a307cda-1bef-4a68-87a8-d5156d230c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:35:17.464807 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1191/2376] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:35:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:35:17.475835 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56d29f-1017-4170-a050-ee22169434a5 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:35:17.478042 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7d56d29f-1017-4170-a050-ee22169434a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:35:17.534059 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d56d29f-1017-4170-a050-ee22169434a5 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:35:17.538129 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b4e2f-f3c9-46c4-8992-aebb35d18fd0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b555bb94ce44ce918cbf2a012576a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:35:17.538588 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b4e2f-f3c9-46c4-8992-aebb35d18fd0 service glance] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:35:17.538929 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b4e2f-f3c9-46c4-8992-aebb35d18fd0 service glance] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:35:17.539811 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b4e2f-f3c9-46c4-8992-aebb35d18fd0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:35:17.548392 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b4e2f-f3c9-46c4-8992-aebb35d18fd0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:35:17.549807 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b4e2f-f3c9-46c4-8992-aebb35d18fd0 service glance] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:35:17.558499 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1186/2377] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:35:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:35:20.313342 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a307cda-1bef-4a68-87a8-d5156d230c9b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:35:20.320051 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0a307cda-1bef-4a68-87a8-d5156d230c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:35:20.375440 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a307cda-1bef-4a68-87a8-d5156d230c9b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:35:20.379165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa3212-3186-4ae8-98c9-4941436a156d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:35:20.379770 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa3212-3186-4ae8-98c9-4941436a156d service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:35:20.380048 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa3212-3186-4ae8-98c9-4941436a156d service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:35:20.380318 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa3212-3186-4ae8-98c9-4941436a156d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:35:20.382036 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4caa3212-3186-4ae8-98c9-4941436a156d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:35:20.432477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:35:20.446381 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa3212-3186-4ae8-98c9-4941436a156d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:35:20.447867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa3212-3186-4ae8-98c9-4941436a156d service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:35:20.457076 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1192/2378] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:35:20 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:35:27.554531 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=6,keystone:SELECT=67,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:35:30.488251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:35:47.617322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b4e2f-f3c9-46c4-8992-aebb35d18fd0 service glance] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:35:47.618810 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ea0b4e2f-f3c9-46c4-8992-aebb35d18fd0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:35:47.691068 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b4e2f-f3c9-46c4-8992-aebb35d18fd0 service glance] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:35:47.696216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221fe17c-ba5a-44aa-98b1-f49e74aa77e0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5177fa4d168c4c219e09c7178bbbc790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:35:47.696974 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221fe17c-ba5a-44aa-98b1-f49e74aa77e0 service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:35:47.697300 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221fe17c-ba5a-44aa-98b1-f49e74aa77e0 service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:35:47.697689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221fe17c-ba5a-44aa-98b1-f49e74aa77e0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:35:47.699477 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-221fe17c-ba5a-44aa-98b1-f49e74aa77e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:35:47.712180 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221fe17c-ba5a-44aa-98b1-f49e74aa77e0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:35:47.713799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221fe17c-ba5a-44aa-98b1-f49e74aa77e0 service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:35:47.726017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1187/2379] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:35:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:35:48.495855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f408d-3f6a-4c54-9d09-6ebb43c4ce1c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:35:48.496484 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f408d-3f6a-4c54-9d09-6ebb43c4ce1c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:35:48.496484 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f408d-3f6a-4c54-9d09-6ebb43c4ce1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:35:48.517914 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-648f408d-3f6a-4c54-9d09-6ebb43c4ce1c None None] MFA Rules not processed for user `d935a4a30e4d44e0a392596452a5c1b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:35:48.519360 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-648f408d-3f6a-4c54-9d09-6ebb43c4ce1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:35:48.590888 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1193/2380] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:35:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:35:57.721261 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=32 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:35:58.572349 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=25 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:08.501211 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221fe17c-ba5a-44aa-98b1-f49e74aa77e0 service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:08.503517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-221fe17c-ba5a-44aa-98b1-f49e74aa77e0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:08.573585 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221fe17c-ba5a-44aa-98b1-f49e74aa77e0 service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:08.578101 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e151e89-0b99-4998-9b47-af54cb713473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:08.578772 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e151e89-0b99-4998-9b47-af54cb713473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:08.579062 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e151e89-0b99-4998-9b47-af54cb713473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:08.579338 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e151e89-0b99-4998-9b47-af54cb713473 admin admin] PATH_INFO: `/v3/users/41fb6ec548094f6eb60714d4da802146` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:08.580462 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e151e89-0b99-4998-9b47-af54cb713473 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41fb6ec548094f6eb60714d4da802146)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:08.582960 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e151e89-0b99-4998-9b47-af54cb713473 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:08.813739 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0e151e89-0b99-4998-9b47-af54cb713473 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41fb6ec548094f6eb60714d4da802146'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:08.901985 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0e151e89-0b99-4998-9b47-af54cb713473 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41fb6ec548094f6eb60714d4da802146'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:08.902733 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0e151e89-0b99-4998-9b47-af54cb713473 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41fb6ec548094f6eb60714d4da802146'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:08.903353 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0e151e89-0b99-4998-9b47-af54cb713473 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41fb6ec548094f6eb60714d4da802146'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:08.916552 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0e151e89-0b99-4998-9b47-af54cb713473 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41fb6ec548094f6eb60714d4da802146'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:08.932737 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1188/2381] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:36:08 2022] DELETE /identity/v3/users/41fb6ec548094f6eb60714d4da802146 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:36:08.938104 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57270278-9ef7-463e-813d-c4504f6c3e99 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:08.938104 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57270278-9ef7-463e-813d-c4504f6c3e99 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:08.938104 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57270278-9ef7-463e-813d-c4504f6c3e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:08.958041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-57270278-9ef7-463e-813d-c4504f6c3e99 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:08.959267 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-57270278-9ef7-463e-813d-c4504f6c3e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:09.027413 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1194/2382] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:36:09.048782 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e151e89-0b99-4998-9b47-af54cb713473 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:09.048782 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0e151e89-0b99-4998-9b47-af54cb713473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:09.109316 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e151e89-0b99-4998-9b47-af54cb713473 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:09.113726 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac6b0cf-f649-43a2-baa7-52d2f7349062 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:09.119392 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6b0cf-f649-43a2-baa7-52d2f7349062 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:09.119392 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6b0cf-f649-43a2-baa7-52d2f7349062 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:09.119392 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6b0cf-f649-43a2-baa7-52d2f7349062 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:09.132887 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac6b0cf-f649-43a2-baa7-52d2f7349062 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:09.134382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac6b0cf-f649-43a2-baa7-52d2f7349062 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:09.144655 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1189/2383] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:36:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:36:09.585173 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57270278-9ef7-463e-813d-c4504f6c3e99 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:09.586299 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-57270278-9ef7-463e-813d-c4504f6c3e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:09.601073 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57270278-9ef7-463e-813d-c4504f6c3e99 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:09.620170 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7726e64-2728-47fe-a440-574896da4f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:09.622056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7726e64-2728-47fe-a440-574896da4f94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:09.622328 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7726e64-2728-47fe-a440-574896da4f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:09.622615 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7726e64-2728-47fe-a440-574896da4f94 admin admin] PATH_INFO: `/v3/projects/017244586132424d96a0dcb3d7d0828a` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:09.625800 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7726e64-2728-47fe-a440-574896da4f94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=017244586132424d96a0dcb3d7d0828a)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:09.627755 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7726e64-2728-47fe-a440-574896da4f94 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:09.638856 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99167058-ee98-40cc-a1c7-0e8d8a6838e1 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:09.639833 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99167058-ee98-40cc-a1c7-0e8d8a6838e1 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:09.640289 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99167058-ee98-40cc-a1c7-0e8d8a6838e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:09.673822 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-99167058-ee98-40cc-a1c7-0e8d8a6838e1 None None] MFA Rules not processed for user `5233689a53364258b86ad2fa012d8db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:09.673822 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-99167058-ee98-40cc-a1c7-0e8d8a6838e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:09.688857 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:09.729448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1190/2384] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Sep 2 09:36:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:36:09.764522 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b7726e64-2728-47fe-a440-574896da4f94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '017244586132424d96a0dcb3d7d0828a', 'request_id': 'req-b7726e64-2728-47fe-a440-574896da4f94'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:09.779954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99167058-ee98-40cc-a1c7-0e8d8a6838e1 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:09.781374 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-99167058-ee98-40cc-a1c7-0e8d8a6838e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:09.841010 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b7726e64-2728-47fe-a440-574896da4f94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '017244586132424d96a0dcb3d7d0828a', 'request_id': 'req-b7726e64-2728-47fe-a440-574896da4f94'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:09.851926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b7726e64-2728-47fe-a440-574896da4f94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '017244586132424d96a0dcb3d7d0828a', 'request_id': 'req-b7726e64-2728-47fe-a440-574896da4f94'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:09.856517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99167058-ee98-40cc-a1c7-0e8d8a6838e1 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:09.863687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b7726e64-2728-47fe-a440-574896da4f94 admin admin] The token cache is being invalidate because project 017244586132424d96a0dcb3d7d0828a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:36:09.864169 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b7726e64-2728-47fe-a440-574896da4f94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:09.864883 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b7726e64-2728-47fe-a440-574896da4f94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:09.866416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1195/2385] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:36:09 2022] DELETE /identity/v3/projects/017244586132424d96a0dcb3d7d0828a => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:36:09.871413 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6831d-c01a-43a2-8166-b5932f2fe17b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:09.871897 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6831d-c01a-43a2-8166-b5932f2fe17b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:09.872390 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6831d-c01a-43a2-8166-b5932f2fe17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:09.892895 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-31e6831d-c01a-43a2-8166-b5932f2fe17b None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:09.894105 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-31e6831d-c01a-43a2-8166-b5932f2fe17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:09.921736 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657c406-80f6-442a-b31f-79997d28f6da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30fe233a7d4c4fbea6ede233bd9ff8c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:09.922409 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657c406-80f6-442a-b31f-79997d28f6da service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:09.922699 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657c406-80f6-442a-b31f-79997d28f6da service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:09.923002 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657c406-80f6-442a-b31f-79997d28f6da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:09.924873 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5657c406-80f6-442a-b31f-79997d28f6da service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:09.971674 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1196/2386] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:36:09.977530 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b73a37-9cea-4585-a39b-66c719b40b8d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:09.977530 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b73a37-9cea-4585-a39b-66c719b40b8d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:09.977530 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b73a37-9cea-4585-a39b-66c719b40b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:09.993619 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c5b73a37-9cea-4585-a39b-66c719b40b8d None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:09.994847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c5b73a37-9cea-4585-a39b-66c719b40b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:09.997444 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657c406-80f6-442a-b31f-79997d28f6da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:09.999511 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657c406-80f6-442a-b31f-79997d28f6da service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:10.003280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1197/2387] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:36:10.007565 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b73a37-9cea-4585-a39b-66c719b40b8d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:10.008038 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1191/2388] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Sep 2 09:36:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 02 09:36:10.017074 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b73a37-9cea-4585-a39b-66c719b40b8d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:10.022933 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0456f2-82d7-40cb-9d6d-c505200ad269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:10.023530 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0456f2-82d7-40cb-9d6d-c505200ad269 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:10.023811 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0456f2-82d7-40cb-9d6d-c505200ad269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:10.024189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0456f2-82d7-40cb-9d6d-c505200ad269 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:10.024775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0456f2-82d7-40cb-9d6d-c505200ad269 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:36:10.025079 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0456f2-82d7-40cb-9d6d-c505200ad269 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:10.026747 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0456f2-82d7-40cb-9d6d-c505200ad269 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:10.039931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1198/2389] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:36:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:36:10.044691 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657c406-80f6-442a-b31f-79997d28f6da service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:10.057618 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657c406-80f6-442a-b31f-79997d28f6da service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:10.062137 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73129aff-2182-4b34-ad58-ac749451a7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:10.062785 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73129aff-2182-4b34-ad58-ac749451a7ca admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:10.063068 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73129aff-2182-4b34-ad58-ac749451a7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:10.063345 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73129aff-2182-4b34-ad58-ac749451a7ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:10.064045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73129aff-2182-4b34-ad58-ac749451a7ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:10.065922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73129aff-2182-4b34-ad58-ac749451a7ca admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:10.077044 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:10.186198 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1192/2390] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:36:10 2022] POST /identity/v3/projects => generated 383 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:36:10.190118 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0456f2-82d7-40cb-9d6d-c505200ad269 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:10.197931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:10.202357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0456f2-82d7-40cb-9d6d-c505200ad269 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:10.282408 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65937f-1244-4c3c-a85f-4da72eb921cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:10.283147 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65937f-1244-4c3c-a85f-4da72eb921cc admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:10.283508 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65937f-1244-4c3c-a85f-4da72eb921cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:10.283858 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65937f-1244-4c3c-a85f-4da72eb921cc admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:10.284650 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d65937f-1244-4c3c-a85f-4da72eb921cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:10.286658 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d65937f-1244-4c3c-a85f-4da72eb921cc admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:10.441030 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1199/2391] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:36:10 2022] POST /identity/v3/users => generated 292 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:36:10.444515 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73129aff-2182-4b34-ad58-ac749451a7ca admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:10.454968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73129aff-2182-4b34-ad58-ac749451a7ca admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:10.460302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f8e0b-f1ba-40b3-83c3-0200078658f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:10.460958 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f8e0b-f1ba-40b3-83c3-0200078658f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:10.461232 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f8e0b-f1ba-40b3-83c3-0200078658f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:10.461502 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f8e0b-f1ba-40b3-83c3-0200078658f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:10.462277 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f8e0b-f1ba-40b3-83c3-0200078658f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:10.463920 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128f8e0b-f1ba-40b3-83c3-0200078658f7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:10.473950 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1193/2392] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:36:10.478324 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65937f-1244-4c3c-a85f-4da72eb921cc admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:10.488714 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65937f-1244-4c3c-a85f-4da72eb921cc admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:10.493375 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4a4d6-956c-411a-86be-b880948fb960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:10.493972 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4a4d6-956c-411a-86be-b880948fb960 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:10.494254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4a4d6-956c-411a-86be-b880948fb960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:10.494528 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4a4d6-956c-411a-86be-b880948fb960 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:10.495212 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4a4d6-956c-411a-86be-b880948fb960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:10.496774 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4a4d6-956c-411a-86be-b880948fb960 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:10.508075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1200/2393] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:36:10.512142 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f8e0b-f1ba-40b3-83c3-0200078658f7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:10.523287 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f8e0b-f1ba-40b3-83c3-0200078658f7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:10.527930 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a325d-87de-4786-b764-ed94e2a058ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:10.528578 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a325d-87de-4786-b764-ed94e2a058ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:10.528861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a325d-87de-4786-b764-ed94e2a058ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:10.529245 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a325d-87de-4786-b764-ed94e2a058ec admin admin] PATH_INFO: `/v3/projects/88c9d4087b544e90adcd37236809fa2e/users/c2d221fb1a2e47279131275f3fd4da4a/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:10.547696 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a325d-87de-4786-b764-ed94e2a058ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=88c9d4087b544e90adcd37236809fa2e, user_id=c2d221fb1a2e47279131275f3fd4da4a, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:10.550441 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a325d-87de-4786-b764-ed94e2a058ec admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:10.634407 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1194/2394] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:36:10 2022] PUT /identity/v3/projects/88c9d4087b544e90adcd37236809fa2e/users/c2d221fb1a2e47279131275f3fd4da4a/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:36:10.645793 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ecf0a-2cc4-48a8-8293-ee019f1ed87e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:10.646081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ecf0a-2cc4-48a8-8293-ee019f1ed87e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:10.646358 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ecf0a-2cc4-48a8-8293-ee019f1ed87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:10.674365 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-205ecf0a-2cc4-48a8-8293-ee019f1ed87e None None] MFA Rules not processed for user `c2d221fb1a2e47279131275f3fd4da4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:10.678208 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-205ecf0a-2cc4-48a8-8293-ee019f1ed87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:10.746545 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1201/2395] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:36:10.755286 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a325d-87de-4786-b764-ed94e2a058ec admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:10.757264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9b7a325d-87de-4786-b764-ed94e2a058ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:10.814294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a325d-87de-4786-b764-ed94e2a058ec admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:10.818917 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb4da3-0b03-4799-b3ac-64d66b3a8ae6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:10.819602 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb4da3-0b03-4799-b3ac-64d66b3a8ae6 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:10.819877 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb4da3-0b03-4799-b3ac-64d66b3a8ae6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:10.820150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb4da3-0b03-4799-b3ac-64d66b3a8ae6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:10.828394 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb4da3-0b03-4799-b3ac-64d66b3a8ae6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:10.829885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb4da3-0b03-4799-b3ac-64d66b3a8ae6 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:10.844744 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1195/2396] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:36:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:36:10.853743 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ecf0a-2cc4-48a8-8293-ee019f1ed87e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:10.855262 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-205ecf0a-2cc4-48a8-8293-ee019f1ed87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:10.932147 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ecf0a-2cc4-48a8-8293-ee019f1ed87e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:10.937493 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16263f-7aa4-4907-bc35-c6eb255b3e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:10.938905 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16263f-7aa4-4907-bc35-c6eb255b3e87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:10.938905 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16263f-7aa4-4907-bc35-c6eb255b3e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:10.938905 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16263f-7aa4-4907-bc35-c6eb255b3e87 admin admin] PATH_INFO: `/v3/users/d2c6655886ed4cde8617b1b4d50be464` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:10.939969 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df16263f-7aa4-4907-bc35-c6eb255b3e87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2c6655886ed4cde8617b1b4d50be464)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:10.941929 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df16263f-7aa4-4907-bc35-c6eb255b3e87 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:10.956975 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:10.993866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67da29-f4fc-456e-b166-3fc65bc7f2f6 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:10.994154 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67da29-f4fc-456e-b166-3fc65bc7f2f6 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:10.994452 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67da29-f4fc-456e-b166-3fc65bc7f2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:11.014307 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4a67da29-f4fc-456e-b166-3fc65bc7f2f6 None None] MFA Rules not processed for user `5233689a53364258b86ad2fa012d8db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:11.015588 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4a67da29-f4fc-456e-b166-3fc65bc7f2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:11.071161 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1196/2397] 10.222.0.57 () {58 vars in 967 bytes} [Fri Sep 2 09:36:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:36:11.083545 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67da29-f4fc-456e-b166-3fc65bc7f2f6 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:11.084843 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4a67da29-f4fc-456e-b166-3fc65bc7f2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:11.140923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-df16263f-7aa4-4907-bc35-c6eb255b3e87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2c6655886ed4cde8617b1b4d50be464'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:11.147879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-df16263f-7aa4-4907-bc35-c6eb255b3e87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2c6655886ed4cde8617b1b4d50be464'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:11.152310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:11.156895 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67da29-f4fc-456e-b166-3fc65bc7f2f6 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:11.162783 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df6dd4-65b8-4f99-a66c-151e85587ac4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30fe233a7d4c4fbea6ede233bd9ff8c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:11.163421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df6dd4-65b8-4f99-a66c-151e85587ac4 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:11.163813 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df6dd4-65b8-4f99-a66c-151e85587ac4 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:11.164093 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df6dd4-65b8-4f99-a66c-151e85587ac4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:11.172171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df6dd4-65b8-4f99-a66c-151e85587ac4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:11.173749 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df6dd4-65b8-4f99-a66c-151e85587ac4 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:11.182267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1197/2398] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Sep 2 09:36:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 02 09:36:11.223674 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-df16263f-7aa4-4907-bc35-c6eb255b3e87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2c6655886ed4cde8617b1b4d50be464'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:11.227095 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-df16263f-7aa4-4907-bc35-c6eb255b3e87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2c6655886ed4cde8617b1b4d50be464'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:11.237029 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-df16263f-7aa4-4907-bc35-c6eb255b3e87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2c6655886ed4cde8617b1b4d50be464'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:11.243425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1202/2399] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:36:10 2022] DELETE /identity/v3/users/d2c6655886ed4cde8617b1b4d50be464 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:36:11.247993 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5f821-f395-4e4f-913b-51d10b4bfced None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:11.248280 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5f821-f395-4e4f-913b-51d10b4bfced None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:11.248557 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5f821-f395-4e4f-913b-51d10b4bfced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:11.267844 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-cdb5f821-f395-4e4f-913b-51d10b4bfced None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:11.269063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cdb5f821-f395-4e4f-913b-51d10b4bfced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:11.329086 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1198/2400] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:36:11.340862 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16263f-7aa4-4907-bc35-c6eb255b3e87 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:11.342545 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-df16263f-7aa4-4907-bc35-c6eb255b3e87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:11.409202 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16263f-7aa4-4907-bc35-c6eb255b3e87 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:11.415649 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a295628-7f78-48bf-8be2-7ea036ad979e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:11.420017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a295628-7f78-48bf-8be2-7ea036ad979e service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:11.420017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a295628-7f78-48bf-8be2-7ea036ad979e service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:11.420017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a295628-7f78-48bf-8be2-7ea036ad979e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:11.431094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a295628-7f78-48bf-8be2-7ea036ad979e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:11.432644 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a295628-7f78-48bf-8be2-7ea036ad979e service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:11.451102 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1203/2401] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:36:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:36:11.916253 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5f821-f395-4e4f-913b-51d10b4bfced None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:11.918245 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cdb5f821-f395-4e4f-913b-51d10b4bfced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:11.931743 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5f821-f395-4e4f-913b-51d10b4bfced None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:11.932609 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad495a23-beaa-46ef-b91e-03508d784d3a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:11.933231 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad495a23-beaa-46ef-b91e-03508d784d3a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:11.933671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad495a23-beaa-46ef-b91e-03508d784d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:11.944578 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a88989-cf84-447d-9ecc-7779874bf04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:11.945230 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a88989-cf84-447d-9ecc-7779874bf04e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:11.945591 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a88989-cf84-447d-9ecc-7779874bf04e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:11.945776 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a88989-cf84-447d-9ecc-7779874bf04e admin admin] PATH_INFO: `/v3/projects/82e9167182ee42e18d58cdd818374c22` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:11.947086 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a88989-cf84-447d-9ecc-7779874bf04e admin admin] RBAC: Authorizing `identity:delete_project(project_id=82e9167182ee42e18d58cdd818374c22)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:11.948932 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a88989-cf84-447d-9ecc-7779874bf04e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:11.954564 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ad495a23-beaa-46ef-b91e-03508d784d3a None None] MFA Rules not processed for user `5233689a53364258b86ad2fa012d8db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:11.955895 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ad495a23-beaa-46ef-b91e-03508d784d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:12.012169 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1204/2402] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Sep 2 09:36:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:36:12.023950 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad495a23-beaa-46ef-b91e-03508d784d3a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:12.025246 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ad495a23-beaa-46ef-b91e-03508d784d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:12.069554 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:12.075591 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c8a88989-cf84-447d-9ecc-7779874bf04e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82e9167182ee42e18d58cdd818374c22', 'request_id': 'req-c8a88989-cf84-447d-9ecc-7779874bf04e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:12.080846 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c8a88989-cf84-447d-9ecc-7779874bf04e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82e9167182ee42e18d58cdd818374c22', 'request_id': 'req-c8a88989-cf84-447d-9ecc-7779874bf04e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:12.087559 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c8a88989-cf84-447d-9ecc-7779874bf04e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82e9167182ee42e18d58cdd818374c22', 'request_id': 'req-c8a88989-cf84-447d-9ecc-7779874bf04e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:12.095767 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad495a23-beaa-46ef-b91e-03508d784d3a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:12.148010 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c8a88989-cf84-447d-9ecc-7779874bf04e admin admin] The token cache is being invalidate because project 82e9167182ee42e18d58cdd818374c22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:36:12.148303 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c8a88989-cf84-447d-9ecc-7779874bf04e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:12.148931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-c8a88989-cf84-447d-9ecc-7779874bf04e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:12.150507 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1199/2403] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:36:11 2022] DELETE /identity/v3/projects/82e9167182ee42e18d58cdd818374c22 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:36:12.155239 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95ff11-d34f-4ba8-ace5-23bcf07a6147 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:12.155520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95ff11-d34f-4ba8-ace5-23bcf07a6147 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:12.155800 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95ff11-d34f-4ba8-ace5-23bcf07a6147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:12.157522 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e577c70-a58a-4cef-a760-7a714dd223b4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30fe233a7d4c4fbea6ede233bd9ff8c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:12.158247 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e577c70-a58a-4cef-a760-7a714dd223b4 service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:12.158593 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e577c70-a58a-4cef-a760-7a714dd223b4 service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:12.159290 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e577c70-a58a-4cef-a760-7a714dd223b4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:12.162422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5e577c70-a58a-4cef-a760-7a714dd223b4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:12.176202 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2a95ff11-d34f-4ba8-ace5-23bcf07a6147 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:12.177417 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2a95ff11-d34f-4ba8-ace5-23bcf07a6147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:12.221601 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e577c70-a58a-4cef-a760-7a714dd223b4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:12.223809 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e577c70-a58a-4cef-a760-7a714dd223b4 service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:12.232214 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1205/2404] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Sep 2 09:36:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 02 09:36:12.243467 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1200/2405] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:36:12.253088 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0813e-baaa-471e-a135-f8a6d5e94773 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:12.253401 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0813e-baaa-471e-a135-f8a6d5e94773 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:12.253674 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b0813e-baaa-471e-a135-f8a6d5e94773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:12.273942 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-11b0813e-baaa-471e-a135-f8a6d5e94773 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:12.275218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-11b0813e-baaa-471e-a135-f8a6d5e94773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:12.283267 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1206/2406] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:36:12.288140 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95ff11-d34f-4ba8-ace5-23bcf07a6147 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:12.297782 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95ff11-d34f-4ba8-ace5-23bcf07a6147 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:12.302361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d72837f-07a8-4ac8-9391-91fe37b5ba05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:12.304613 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d72837f-07a8-4ac8-9391-91fe37b5ba05 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:12.304613 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d72837f-07a8-4ac8-9391-91fe37b5ba05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:12.304613 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d72837f-07a8-4ac8-9391-91fe37b5ba05 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:12.304613 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d72837f-07a8-4ac8-9391-91fe37b5ba05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:36:12.304613 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d72837f-07a8-4ac8-9391-91fe37b5ba05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:12.306002 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d72837f-07a8-4ac8-9391-91fe37b5ba05 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:12.317427 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1201/2407] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:36:12.321684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b0813e-baaa-471e-a135-f8a6d5e94773 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:12.330817 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b0813e-baaa-471e-a135-f8a6d5e94773 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:12.336018 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe62d1e-e764-4532-85a8-9fe92eadc8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:12.336638 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe62d1e-e764-4532-85a8-9fe92eadc8fd admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:12.336915 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe62d1e-e764-4532-85a8-9fe92eadc8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:12.337182 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe62d1e-e764-4532-85a8-9fe92eadc8fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:12.337798 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe62d1e-e764-4532-85a8-9fe92eadc8fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:12.339883 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe62d1e-e764-4532-85a8-9fe92eadc8fd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:12.452956 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1207/2408] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:36:12 2022] POST /identity/v3/projects => generated 377 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:36:12.457859 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d72837f-07a8-4ac8-9391-91fe37b5ba05 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:12.463007 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:12.468228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d72837f-07a8-4ac8-9391-91fe37b5ba05 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:12.527166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50110733-8010-4844-a4f1-350961ba4237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:12.528403 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50110733-8010-4844-a4f1-350961ba4237 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:12.528771 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50110733-8010-4844-a4f1-350961ba4237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:12.528965 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50110733-8010-4844-a4f1-350961ba4237 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:12.529608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50110733-8010-4844-a4f1-350961ba4237 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:12.531904 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50110733-8010-4844-a4f1-350961ba4237 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:12.622243 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1202/2409] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:36:12 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:36:12.624726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe62d1e-e764-4532-85a8-9fe92eadc8fd admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:12.639612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe62d1e-e764-4532-85a8-9fe92eadc8fd admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:12.645734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60abb5-5206-4f7d-a9a8-8b45aafe7630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:12.646336 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60abb5-5206-4f7d-a9a8-8b45aafe7630 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:12.646627 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60abb5-5206-4f7d-a9a8-8b45aafe7630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:12.646938 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c60abb5-5206-4f7d-a9a8-8b45aafe7630 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:12.647736 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c60abb5-5206-4f7d-a9a8-8b45aafe7630 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:12.649548 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c60abb5-5206-4f7d-a9a8-8b45aafe7630 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:12.663142 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1208/2410] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:36:12.670920 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50110733-8010-4844-a4f1-350961ba4237 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:12.689124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50110733-8010-4844-a4f1-350961ba4237 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:12.698129 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0b067-fc86-4ef4-9d72-9c95b1571bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:12.698820 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0b067-fc86-4ef4-9d72-9c95b1571bd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:12.699146 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0b067-fc86-4ef4-9d72-9c95b1571bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:12.699701 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a0b067-fc86-4ef4-9d72-9c95b1571bd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:12.700087 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a0b067-fc86-4ef4-9d72-9c95b1571bd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:12.701698 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a0b067-fc86-4ef4-9d72-9c95b1571bd9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:12.720206 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1203/2411] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:36:12.723824 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60abb5-5206-4f7d-a9a8-8b45aafe7630 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:12.736573 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c60abb5-5206-4f7d-a9a8-8b45aafe7630 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:12.742797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e2bcb8-c7cb-484c-89be-28851950b104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:12.743492 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2bcb8-c7cb-484c-89be-28851950b104 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:12.743781 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2bcb8-c7cb-484c-89be-28851950b104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:12.748822 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e2bcb8-c7cb-484c-89be-28851950b104 admin admin] PATH_INFO: `/v3/projects/0496f560580c4fd59ff85eaab508c538/users/343bf9821bf84c23a11b8f6d15c49a9c/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:12.752032 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0b067-fc86-4ef4-9d72-9c95b1571bd9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:12.756206 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-80a0b067-fc86-4ef4-9d72-9c95b1571bd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:12.761358 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e2bcb8-c7cb-484c-89be-28851950b104 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0496f560580c4fd59ff85eaab508c538, user_id=343bf9821bf84c23a11b8f6d15c49a9c, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:12.764506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e2bcb8-c7cb-484c-89be-28851950b104 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:12.772129 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a0b067-fc86-4ef4-9d72-9c95b1571bd9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:12.776807 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb147d-f3b6-44a6-9ec9-f3d3afaa3a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:12.777423 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb147d-f3b6-44a6-9ec9-f3d3afaa3a25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:12.777699 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb147d-f3b6-44a6-9ec9-f3d3afaa3a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:12.777977 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb147d-f3b6-44a6-9ec9-f3d3afaa3a25 admin admin] PATH_INFO: `/v3/users/3e103dd2c68242499a8c7eae41b9d183` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:12.779126 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb147d-f3b6-44a6-9ec9-f3d3afaa3a25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e103dd2c68242499a8c7eae41b9d183)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:12.781025 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb147d-f3b6-44a6-9ec9-f3d3afaa3a25 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:12.863491 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1209/2412] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:36:12 2022] PUT /identity/v3/projects/0496f560580c4fd59ff85eaab508c538/users/343bf9821bf84c23a11b8f6d15c49a9c/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:36:12.876991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a9a4b-c931-46d4-91e2-1ba41b3a4623 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:12.877928 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a9a4b-c931-46d4-91e2-1ba41b3a4623 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:12.878750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a9a4b-c931-46d4-91e2-1ba41b3a4623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:12.907967 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8f4a9a4b-c931-46d4-91e2-1ba41b3a4623 None None] MFA Rules not processed for user `343bf9821bf84c23a11b8f6d15c49a9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:12.909186 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8f4a9a4b-c931-46d4-91e2-1ba41b3a4623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:12.917744 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:12.945800 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d4bb147d-f3b6-44a6-9ec9-f3d3afaa3a25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e103dd2c68242499a8c7eae41b9d183'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:12.977877 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1210/2413] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:36:12.979061 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00c43e-ef64-463d-9850-1f8d460e6ae2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:12.979434 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00c43e-ef64-463d-9850-1f8d460e6ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:12.979707 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00c43e-ef64-463d-9850-1f8d460e6ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:12.998510 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8d00c43e-ef64-463d-9850-1f8d460e6ae2 None None] MFA Rules not processed for user `5233689a53364258b86ad2fa012d8db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:13.000002 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8d00c43e-ef64-463d-9850-1f8d460e6ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:13.011481 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d4bb147d-f3b6-44a6-9ec9-f3d3afaa3a25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e103dd2c68242499a8c7eae41b9d183'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:13.012152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d4bb147d-f3b6-44a6-9ec9-f3d3afaa3a25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e103dd2c68242499a8c7eae41b9d183'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:13.012824 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d4bb147d-f3b6-44a6-9ec9-f3d3afaa3a25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e103dd2c68242499a8c7eae41b9d183'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:13.025990 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d4bb147d-f3b6-44a6-9ec9-f3d3afaa3a25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e103dd2c68242499a8c7eae41b9d183'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:13.039438 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1204/2414] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:36:12 2022] DELETE /identity/v3/users/3e103dd2c68242499a8c7eae41b9d183 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:36:13.042632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb147d-f3b6-44a6-9ec9-f3d3afaa3a25 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:13.042632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d4bb147d-f3b6-44a6-9ec9-f3d3afaa3a25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:13.054620 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1211/2415] 10.222.0.57 () {58 vars in 967 bytes} [Fri Sep 2 09:36:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:36:13.056015 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5e6f0-3471-45c7-bf56-caf637a7ce5d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:13.056296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5e6f0-3471-45c7-bf56-caf637a7ce5d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:13.056566 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5e6f0-3471-45c7-bf56-caf637a7ce5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:13.075602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3ea5e6f0-3471-45c7-bf56-caf637a7ce5d None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:13.076918 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3ea5e6f0-3471-45c7-bf56-caf637a7ce5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:13.101515 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:13.113084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb147d-f3b6-44a6-9ec9-f3d3afaa3a25 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:13.117813 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c82e96-cb7d-4a85-b74a-ee2b6c4d509f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:13.118416 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c82e96-cb7d-4a85-b74a-ee2b6c4d509f admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:13.118760 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c82e96-cb7d-4a85-b74a-ee2b6c4d509f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:13.119041 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c82e96-cb7d-4a85-b74a-ee2b6c4d509f admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:13.119661 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c82e96-cb7d-4a85-b74a-ee2b6c4d509f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:13.121560 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c82e96-cb7d-4a85-b74a-ee2b6c4d509f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:13.139962 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1212/2416] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:36:13.141457 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5e6f0-3471-45c7-bf56-caf637a7ce5d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:13.142301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3ea5e6f0-3471-45c7-bf56-caf637a7ce5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:13.193124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1205/2417] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:36:13 2022] POST /identity/v3/projects => generated 377 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:36:13.193124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c82e96-cb7d-4a85-b74a-ee2b6c4d509f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:13.193124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-62c82e96-cb7d-4a85-b74a-ee2b6c4d509f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:13.199869 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:13.215014 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5e6f0-3471-45c7-bf56-caf637a7ce5d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:13.248741 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c82e96-cb7d-4a85-b74a-ee2b6c4d509f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:13.252623 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d36ae-a54c-4dea-973d-5e86ee4da207 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:13.253225 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d36ae-a54c-4dea-973d-5e86ee4da207 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:13.253500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d36ae-a54c-4dea-973d-5e86ee4da207 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:13.253768 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d36ae-a54c-4dea-973d-5e86ee4da207 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:13.274337 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8c425-e7fe-4a09-a506-1af7b3c528f8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30fe233a7d4c4fbea6ede233bd9ff8c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:13.274974 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8c425-e7fe-4a09-a506-1af7b3c528f8 service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:13.275270 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8c425-e7fe-4a09-a506-1af7b3c528f8 service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:13.275539 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8c425-e7fe-4a09-a506-1af7b3c528f8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:13.319204 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d36ae-a54c-4dea-973d-5e86ee4da207 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:13.320652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d36ae-a54c-4dea-973d-5e86ee4da207 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:13.330416 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1206/2418] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:36:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:36:13.331542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d36ae-a54c-4dea-973d-5e86ee4da207 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:13.336148 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8c425-e7fe-4a09-a506-1af7b3c528f8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:13.338245 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8c425-e7fe-4a09-a506-1af7b3c528f8 service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:13.341867 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d36ae-a54c-4dea-973d-5e86ee4da207 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:13.347138 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64a245-d259-4545-9310-96c6604de5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:13.347753 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64a245-d259-4545-9310-96c6604de5b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:13.348095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64a245-d259-4545-9310-96c6604de5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:13.348372 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64a245-d259-4545-9310-96c6604de5b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:13.349089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a64a245-d259-4545-9310-96c6604de5b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:13.349957 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1213/2419] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Sep 2 09:36:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 09:36:13.352124 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a64a245-d259-4545-9310-96c6604de5b8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:13.426741 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1207/2420] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:36:13 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:36:13.428853 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8c425-e7fe-4a09-a506-1af7b3c528f8 service swift] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:13.440267 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8c425-e7fe-4a09-a506-1af7b3c528f8 service swift] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:13.445458 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f662f98-6797-4a43-9849-ea33be556a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:13.446058 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f662f98-6797-4a43-9849-ea33be556a22 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:13.446340 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f662f98-6797-4a43-9849-ea33be556a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:13.446616 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f662f98-6797-4a43-9849-ea33be556a22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:13.447295 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f662f98-6797-4a43-9849-ea33be556a22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:13.448907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f662f98-6797-4a43-9849-ea33be556a22 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:13.459758 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1214/2421] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:36:13.463686 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64a245-d259-4545-9310-96c6604de5b8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:13.473649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64a245-d259-4545-9310-96c6604de5b8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:13.478159 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffed8fd-d59e-4177-9739-8c4fd11c70c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:13.478806 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffed8fd-d59e-4177-9739-8c4fd11c70c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:13.479092 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffed8fd-d59e-4177-9739-8c4fd11c70c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:13.479364 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffed8fd-d59e-4177-9739-8c4fd11c70c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:13.480003 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffed8fd-d59e-4177-9739-8c4fd11c70c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:13.481615 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffed8fd-d59e-4177-9739-8c4fd11c70c8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:13.492570 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1208/2422] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:36:13.502160 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f662f98-6797-4a43-9849-ea33be556a22 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:13.512894 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f662f98-6797-4a43-9849-ea33be556a22 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:13.517555 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f5afa2-760c-40f2-a6a7-6db7d2508db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:13.518218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5afa2-760c-40f2-a6a7-6db7d2508db9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:13.518557 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5afa2-760c-40f2-a6a7-6db7d2508db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:13.518868 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f5afa2-760c-40f2-a6a7-6db7d2508db9 admin admin] PATH_INFO: `/v3/projects/89c677fdd5ed4b0981c9064997f600ba/users/f48c58f4ed2343d89776cadf3983551d/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:13.536978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f5afa2-760c-40f2-a6a7-6db7d2508db9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89c677fdd5ed4b0981c9064997f600ba, user_id=f48c58f4ed2343d89776cadf3983551d, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:13.539772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f5afa2-760c-40f2-a6a7-6db7d2508db9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:13.600308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1215/2423] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:36:13 2022] PUT /identity/v3/projects/89c677fdd5ed4b0981c9064997f600ba/users/f48c58f4ed2343d89776cadf3983551d/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:36:13.610349 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d2154-b24c-4b1f-bb3d-e0923ea4a3c2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:13.610634 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d2154-b24c-4b1f-bb3d-e0923ea4a3c2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:13.610957 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d2154-b24c-4b1f-bb3d-e0923ea4a3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:13.639925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-225d2154-b24c-4b1f-bb3d-e0923ea4a3c2 None None] MFA Rules not processed for user `f48c58f4ed2343d89776cadf3983551d`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:13.641151 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-225d2154-b24c-4b1f-bb3d-e0923ea4a3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:13.686700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:13.706605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1209/2424] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:36:13.711924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f5afa2-760c-40f2-a6a7-6db7d2508db9 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:13.734285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d2154-b24c-4b1f-bb3d-e0923ea4a3c2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:13.739043 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-225d2154-b24c-4b1f-bb3d-e0923ea4a3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:13.842674 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d2154-b24c-4b1f-bb3d-e0923ea4a3c2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:13.857555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b425477-85f0-43cd-8e41-a8138f09cfd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:13.863166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b425477-85f0-43cd-8e41-a8138f09cfd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:13.863495 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b425477-85f0-43cd-8e41-a8138f09cfd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:13.864300 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b425477-85f0-43cd-8e41-a8138f09cfd8 admin admin] PATH_INFO: `/v3/projects/a74a8dc588bf4dfe9fb9165ffe948e8c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:13.866589 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b425477-85f0-43cd-8e41-a8138f09cfd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a74a8dc588bf4dfe9fb9165ffe948e8c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:13.868840 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b425477-85f0-43cd-8e41-a8138f09cfd8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:13.875806 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f5afa2-760c-40f2-a6a7-6db7d2508db9 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:13.883552 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da54220-b7c2-4c94-95a2-f027df5d830d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:13.884176 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da54220-b7c2-4c94-95a2-f027df5d830d admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:13.884453 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da54220-b7c2-4c94-95a2-f027df5d830d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:13.884732 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da54220-b7c2-4c94-95a2-f027df5d830d admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:13.885467 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da54220-b7c2-4c94-95a2-f027df5d830d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:13.890519 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da54220-b7c2-4c94-95a2-f027df5d830d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:13.904015 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:14.021932 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3b425477-85f0-43cd-8e41-a8138f09cfd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a74a8dc588bf4dfe9fb9165ffe948e8c', 'request_id': 'req-3b425477-85f0-43cd-8e41-a8138f09cfd8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:14.030193 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3b425477-85f0-43cd-8e41-a8138f09cfd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a74a8dc588bf4dfe9fb9165ffe948e8c', 'request_id': 'req-3b425477-85f0-43cd-8e41-a8138f09cfd8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:14.038251 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3b425477-85f0-43cd-8e41-a8138f09cfd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a74a8dc588bf4dfe9fb9165ffe948e8c', 'request_id': 'req-3b425477-85f0-43cd-8e41-a8138f09cfd8'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:14.038857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1216/2425] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:36:13 2022] POST /identity/v3/projects => generated 373 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:36:14.042834 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da54220-b7c2-4c94-95a2-f027df5d830d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:14.106236 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da54220-b7c2-4c94-95a2-f027df5d830d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:14.110942 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da2a4c-a901-49b5-a3b9-a0fb9ffccedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:14.111550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da2a4c-a901-49b5-a3b9-a0fb9ffccedb admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:14.111830 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da2a4c-a901-49b5-a3b9-a0fb9ffccedb admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:14.112103 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7da2a4c-a901-49b5-a3b9-a0fb9ffccedb admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:14.112764 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da2a4c-a901-49b5-a3b9-a0fb9ffccedb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:14.114672 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7da2a4c-a901-49b5-a3b9-a0fb9ffccedb admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:14.123817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3b425477-85f0-43cd-8e41-a8138f09cfd8 admin admin] The token cache is being invalidate because project a74a8dc588bf4dfe9fb9165ffe948e8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:36:14.124199 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3b425477-85f0-43cd-8e41-a8138f09cfd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:14.125367 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3b425477-85f0-43cd-8e41-a8138f09cfd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:14.127514 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1210/2426] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:36:13 2022] DELETE /identity/v3/projects/a74a8dc588bf4dfe9fb9165ffe948e8c => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:36:14.132882 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b425477-85f0-43cd-8e41-a8138f09cfd8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:14.135544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3b425477-85f0-43cd-8e41-a8138f09cfd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:14.210804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b425477-85f0-43cd-8e41-a8138f09cfd8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:14.214767 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1217/2427] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:36:14 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:36:14.215579 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3809d5-e28a-4846-948a-2641a9695909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:14.216219 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3809d5-e28a-4846-948a-2641a9695909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:14.216503 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3809d5-e28a-4846-948a-2641a9695909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:14.216779 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3809d5-e28a-4846-948a-2641a9695909 admin admin] PATH_INFO: `/v3/users/7eb17135812e48c68c11eb0d82e50e43` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:14.217943 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3809d5-e28a-4846-948a-2641a9695909 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7eb17135812e48c68c11eb0d82e50e43)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:14.218509 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da2a4c-a901-49b5-a3b9-a0fb9ffccedb admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:14.219897 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3809d5-e28a-4846-948a-2641a9695909 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:14.220346 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f7da2a4c-a901-49b5-a3b9-a0fb9ffccedb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:14.234253 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7da2a4c-a901-49b5-a3b9-a0fb9ffccedb admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:14.242442 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58f882-1537-4f4f-b0f3-5f99859c723a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:14.243107 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58f882-1537-4f4f-b0f3-5f99859c723a admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:14.243386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58f882-1537-4f4f-b0f3-5f99859c723a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:14.243658 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58f882-1537-4f4f-b0f3-5f99859c723a admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:14.244300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a58f882-1537-4f4f-b0f3-5f99859c723a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:14.245917 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a58f882-1537-4f4f-b0f3-5f99859c723a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:14.257402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1218/2428] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:36:14.261525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58f882-1537-4f4f-b0f3-5f99859c723a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:14.271891 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58f882-1537-4f4f-b0f3-5f99859c723a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:14.276533 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc4d1cf-303e-4b10-bbbc-5a06d2a40aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:14.277175 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4d1cf-303e-4b10-bbbc-5a06d2a40aca admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:14.277457 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4d1cf-303e-4b10-bbbc-5a06d2a40aca admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:14.277734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc4d1cf-303e-4b10-bbbc-5a06d2a40aca admin admin] PATH_INFO: `/v3/projects/7f0f2ffa2c404e8687438d3f5815e7e7/users/fc55015f63c24025973f7416f79bcbff/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:14.297204 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:14.309010 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc4d1cf-303e-4b10-bbbc-5a06d2a40aca admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f0f2ffa2c404e8687438d3f5815e7e7, user_id=fc55015f63c24025973f7416f79bcbff, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:14.311787 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc4d1cf-303e-4b10-bbbc-5a06d2a40aca admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:14.401945 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1219/2429] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:36:14 2022] PUT /identity/v3/projects/7f0f2ffa2c404e8687438d3f5815e7e7/users/fc55015f63c24025973f7416f79bcbff/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:36:14.408338 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc4d1cf-303e-4b10-bbbc-5a06d2a40aca admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:14.476629 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc4d1cf-303e-4b10-bbbc-5a06d2a40aca admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:14.476883 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2d3809d5-e28a-4846-948a-2641a9695909 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7eb17135812e48c68c11eb0d82e50e43'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:14.481266 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7f4d4-f1d2-49dc-a69e-db81ae40ff67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:14.481904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7f4d4-f1d2-49dc-a69e-db81ae40ff67 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:14.482186 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7f4d4-f1d2-49dc-a69e-db81ae40ff67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:14.482459 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7f4d4-f1d2-49dc-a69e-db81ae40ff67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:14.483143 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7f4d4-f1d2-49dc-a69e-db81ae40ff67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:14.484724 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7f4d4-f1d2-49dc-a69e-db81ae40ff67 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:14.494971 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1220/2430] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:36:14.499087 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7f4d4-f1d2-49dc-a69e-db81ae40ff67 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:14.509467 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7f4d4-f1d2-49dc-a69e-db81ae40ff67 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:14.514211 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8d68d-4648-4e7f-8b97-92a36e68bd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:14.514810 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8d68d-4648-4e7f-8b97-92a36e68bd73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:14.515098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8d68d-4648-4e7f-8b97-92a36e68bd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:14.515395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8d68d-4648-4e7f-8b97-92a36e68bd73 admin admin] PATH_INFO: `/v3/projects/7f0f2ffa2c404e8687438d3f5815e7e7/users/fc55015f63c24025973f7416f79bcbff/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:14.517264 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8d68d-4648-4e7f-8b97-92a36e68bd73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f0f2ffa2c404e8687438d3f5815e7e7, user_id=fc55015f63c24025973f7416f79bcbff, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:14.519989 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8d68d-4648-4e7f-8b97-92a36e68bd73 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:14.530050 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1221/2431] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:36:14 2022] PUT /identity/v3/projects/7f0f2ffa2c404e8687438d3f5815e7e7/users/fc55015f63c24025973f7416f79bcbff/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:36:14.540753 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf0db7-cba8-45d3-bac5-60cac31f1c4b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:14.541042 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf0db7-cba8-45d3-bac5-60cac31f1c4b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:14.541320 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf0db7-cba8-45d3-bac5-60cac31f1c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:14.549247 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2d3809d5-e28a-4846-948a-2641a9695909 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7eb17135812e48c68c11eb0d82e50e43'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:14.550265 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2d3809d5-e28a-4846-948a-2641a9695909 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7eb17135812e48c68c11eb0d82e50e43'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:14.550917 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2d3809d5-e28a-4846-948a-2641a9695909 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7eb17135812e48c68c11eb0d82e50e43'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:14.564714 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-2d3809d5-e28a-4846-948a-2641a9695909 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7eb17135812e48c68c11eb0d82e50e43'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:14.577757 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-43cf0db7-cba8-45d3-bac5-60cac31f1c4b None None] MFA Rules not processed for user `fc55015f63c24025973f7416f79bcbff`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:14.579030 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-43cf0db7-cba8-45d3-bac5-60cac31f1c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:14.597873 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1211/2432] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:36:14 2022] DELETE /identity/v3/users/7eb17135812e48c68c11eb0d82e50e43 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:36:14.598290 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:14.657686 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1222/2433] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:36:14.671380 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3809d5-e28a-4846-948a-2641a9695909 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:14.672977 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2d3809d5-e28a-4846-948a-2641a9695909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:14.762496 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3809d5-e28a-4846-948a-2641a9695909 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:14.769492 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a84d6-a9f3-4455-82f6-36dc65147e09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:14.769492 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a84d6-a9f3-4455-82f6-36dc65147e09 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:14.769492 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a84d6-a9f3-4455-82f6-36dc65147e09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:14.769492 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a84d6-a9f3-4455-82f6-36dc65147e09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:14.769895 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f25a84d6-a9f3-4455-82f6-36dc65147e09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:14.865498 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a84d6-a9f3-4455-82f6-36dc65147e09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:14.866948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a84d6-a9f3-4455-82f6-36dc65147e09 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:14.878001 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1212/2434] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:36:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:36:15.064558 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf0db7-cba8-45d3-bac5-60cac31f1c4b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:15.066254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-43cf0db7-cba8-45d3-bac5-60cac31f1c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:15.169070 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf0db7-cba8-45d3-bac5-60cac31f1c4b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:15.201513 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a84d6-a9f3-4455-82f6-36dc65147e09 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:15.203166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f25a84d6-a9f3-4455-82f6-36dc65147e09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:15.231848 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b86826a-7b9c-4f4a-98c9-50b790b42e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:15.252496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86826a-7b9c-4f4a-98c9-50b790b42e7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:15.252496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86826a-7b9c-4f4a-98c9-50b790b42e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:15.252496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b86826a-7b9c-4f4a-98c9-50b790b42e7c admin admin] PATH_INFO: `/v3/projects/d553fa0200ef453e846ab02f3a3cdd3e` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:15.252496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b86826a-7b9c-4f4a-98c9-50b790b42e7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d553fa0200ef453e846ab02f3a3cdd3e)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:15.252496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b86826a-7b9c-4f4a-98c9-50b790b42e7c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:15.258765 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:15.271813 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a84d6-a9f3-4455-82f6-36dc65147e09 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:15.276429 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57aa16a-ae32-494b-b005-4180af334c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:15.277046 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57aa16a-ae32-494b-b005-4180af334c4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:15.277322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57aa16a-ae32-494b-b005-4180af334c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:15.277620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57aa16a-ae32-494b-b005-4180af334c4e admin admin] PATH_INFO: `/v3/users/c2d221fb1a2e47279131275f3fd4da4a` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:15.278777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57aa16a-ae32-494b-b005-4180af334c4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2d221fb1a2e47279131275f3fd4da4a)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:15.280718 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57aa16a-ae32-494b-b005-4180af334c4e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:15.397538 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1b86826a-7b9c-4f4a-98c9-50b790b42e7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd553fa0200ef453e846ab02f3a3cdd3e', 'request_id': 'req-1b86826a-7b9c-4f4a-98c9-50b790b42e7c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:15.468131 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1b86826a-7b9c-4f4a-98c9-50b790b42e7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd553fa0200ef453e846ab02f3a3cdd3e', 'request_id': 'req-1b86826a-7b9c-4f4a-98c9-50b790b42e7c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:15.472725 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:15.476623 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1b86826a-7b9c-4f4a-98c9-50b790b42e7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd553fa0200ef453e846ab02f3a3cdd3e', 'request_id': 'req-1b86826a-7b9c-4f4a-98c9-50b790b42e7c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:15.486078 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1b86826a-7b9c-4f4a-98c9-50b790b42e7c admin admin] The token cache is being invalidate because project d553fa0200ef453e846ab02f3a3cdd3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:36:15.486365 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1b86826a-7b9c-4f4a-98c9-50b790b42e7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:15.487010 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1b86826a-7b9c-4f4a-98c9-50b790b42e7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:15.488486 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1223/2435] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:36:15 2022] DELETE /identity/v3/projects/d553fa0200ef453e846ab02f3a3cdd3e => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:36:15.493752 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3067b-e5ed-402f-9bc4-9dfa91751c09 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:15.494042 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3067b-e5ed-402f-9bc4-9dfa91751c09 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:15.494319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3067b-e5ed-402f-9bc4-9dfa91751c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:15.513151 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d2e3067b-e5ed-402f-9bc4-9dfa91751c09 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:15.514329 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d2e3067b-e5ed-402f-9bc4-9dfa91751c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:15.540126 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d57aa16a-ae32-494b-b005-4180af334c4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2d221fb1a2e47279131275f3fd4da4a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:15.571861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1224/2436] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:36:15.580948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffc2eb-081c-43d8-8ef6-c794f64333a9 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:15.581233 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffc2eb-081c-43d8-8ef6-c794f64333a9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:15.581520 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ffc2eb-081c-43d8-8ef6-c794f64333a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:15.602540 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b9ffc2eb-081c-43d8-8ef6-c794f64333a9 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:15.603794 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b9ffc2eb-081c-43d8-8ef6-c794f64333a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:15.620198 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1225/2437] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:36:15.620198 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ffc2eb-081c-43d8-8ef6-c794f64333a9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:15.625796 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ffc2eb-081c-43d8-8ef6-c794f64333a9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:15.630632 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddc6ac-d76e-41cd-801b-a5c99632b261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:15.631263 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddc6ac-d76e-41cd-801b-a5c99632b261 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:15.631552 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddc6ac-d76e-41cd-801b-a5c99632b261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:15.631832 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddc6ac-d76e-41cd-801b-a5c99632b261 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:15.632659 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eddc6ac-d76e-41cd-801b-a5c99632b261 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:36:15.632970 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eddc6ac-d76e-41cd-801b-a5c99632b261 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:15.634554 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eddc6ac-d76e-41cd-801b-a5c99632b261 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:15.640462 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d57aa16a-ae32-494b-b005-4180af334c4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2d221fb1a2e47279131275f3fd4da4a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:15.641229 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d57aa16a-ae32-494b-b005-4180af334c4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2d221fb1a2e47279131275f3fd4da4a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:15.641929 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d57aa16a-ae32-494b-b005-4180af334c4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2d221fb1a2e47279131275f3fd4da4a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:15.647134 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1226/2438] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:36:15.655984 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d57aa16a-ae32-494b-b005-4180af334c4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2d221fb1a2e47279131275f3fd4da4a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:15.657851 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddc6ac-d76e-41cd-801b-a5c99632b261 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:15.659581 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9eddc6ac-d76e-41cd-801b-a5c99632b261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:15.669201 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1213/2439] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:36:15 2022] DELETE /identity/v3/users/c2d221fb1a2e47279131275f3fd4da4a => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:36:15.672800 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddc6ac-d76e-41cd-801b-a5c99632b261 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:15.673412 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f60bc-a567-4093-b4ea-65f91404ae54 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:15.676221 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f60bc-a567-4093-b4ea-65f91404ae54 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:15.677764 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f60bc-a567-4093-b4ea-65f91404ae54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:15.700049 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f21f60bc-a567-4093-b4ea-65f91404ae54 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:15.701510 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f21f60bc-a567-4093-b4ea-65f91404ae54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:15.737362 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c52741-2dfb-42e8-9951-c051b3227977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:15.738081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c52741-2dfb-42e8-9951-c051b3227977 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:15.738415 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c52741-2dfb-42e8-9951-c051b3227977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:15.738811 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c52741-2dfb-42e8-9951-c051b3227977 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:15.740145 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c52741-2dfb-42e8-9951-c051b3227977 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:15.742232 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c52741-2dfb-42e8-9951-c051b3227977 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:15.776344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1214/2440] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:36:15.786417 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f60bc-a567-4093-b4ea-65f91404ae54 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:15.787774 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f21f60bc-a567-4093-b4ea-65f91404ae54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:15.831062 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1227/2441] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:36:15 2022] POST /identity/v3/projects => generated 359 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:36:15.835786 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c52741-2dfb-42e8-9951-c051b3227977 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:15.854566 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f60bc-a567-4093-b4ea-65f91404ae54 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:15.889063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:15.899722 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:15.904782 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c52741-2dfb-42e8-9951-c051b3227977 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:15.909757 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087cfa80-054d-49bb-9557-dfbd1bb6f301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:15.910364 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cfa80-054d-49bb-9557-dfbd1bb6f301 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:15.910718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cfa80-054d-49bb-9557-dfbd1bb6f301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:15.911017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087cfa80-054d-49bb-9557-dfbd1bb6f301 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:15.911648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087cfa80-054d-49bb-9557-dfbd1bb6f301 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:15.912063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a5277-3e3d-47b7-bd07-d48e47d6cbcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:15.912674 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a5277-3e3d-47b7-bd07-d48e47d6cbcf service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:15.912951 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a5277-3e3d-47b7-bd07-d48e47d6cbcf service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:15.913221 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a5277-3e3d-47b7-bd07-d48e47d6cbcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:15.913757 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087cfa80-054d-49bb-9557-dfbd1bb6f301 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:15.923316 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a5277-3e3d-47b7-bd07-d48e47d6cbcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:15.924847 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a5277-3e3d-47b7-bd07-d48e47d6cbcf service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:15.935231 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1215/2442] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:36:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:36:16.017580 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1228/2443] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:36:15 2022] POST /identity/v3/users => generated 280 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:36:16.021820 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a5277-3e3d-47b7-bd07-d48e47d6cbcf service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:16.031510 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a5277-3e3d-47b7-bd07-d48e47d6cbcf service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:16.038011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea20c7-8a4f-493c-ad97-c297963bd965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:16.038741 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea20c7-8a4f-493c-ad97-c297963bd965 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:16.038956 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea20c7-8a4f-493c-ad97-c297963bd965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:16.039223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea20c7-8a4f-493c-ad97-c297963bd965 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:16.039864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea20c7-8a4f-493c-ad97-c297963bd965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:16.041434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea20c7-8a4f-493c-ad97-c297963bd965 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:16.051321 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1216/2444] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:36:16.055761 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087cfa80-054d-49bb-9557-dfbd1bb6f301 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:16.065431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087cfa80-054d-49bb-9557-dfbd1bb6f301 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:16.070637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd3c426-23a5-4fc5-b450-02b89789ffb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:16.071252 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3c426-23a5-4fc5-b450-02b89789ffb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:16.071529 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3c426-23a5-4fc5-b450-02b89789ffb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:16.071798 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd3c426-23a5-4fc5-b450-02b89789ffb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:16.072428 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd3c426-23a5-4fc5-b450-02b89789ffb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:16.073985 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd3c426-23a5-4fc5-b450-02b89789ffb4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:16.083574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1229/2445] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:36:16.092651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea20c7-8a4f-493c-ad97-c297963bd965 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:16.102600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea20c7-8a4f-493c-ad97-c297963bd965 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:16.107134 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b8e17-1884-4e0b-bfe5-c4f9efe55b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:16.107778 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b8e17-1884-4e0b-bfe5-c4f9efe55b82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:16.108059 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b8e17-1884-4e0b-bfe5-c4f9efe55b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:16.108333 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b8e17-1884-4e0b-bfe5-c4f9efe55b82 admin admin] PATH_INFO: `/v3/projects/70a45f947ad544e188c14845d6b672a5/users/a2e187292e594683a52502b0fda86794/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:16.125184 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b8e17-1884-4e0b-bfe5-c4f9efe55b82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70a45f947ad544e188c14845d6b672a5, user_id=a2e187292e594683a52502b0fda86794, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:16.127826 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b8e17-1884-4e0b-bfe5-c4f9efe55b82 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:16.190147 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd3c426-23a5-4fc5-b450-02b89789ffb4 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:16.192009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4cd3c426-23a5-4fc5-b450-02b89789ffb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:16.205800 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd3c426-23a5-4fc5-b450-02b89789ffb4 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:16.212431 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61843b47-4db6-4122-b6e9-e96f410809ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:16.213025 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61843b47-4db6-4122-b6e9-e96f410809ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:16.213305 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61843b47-4db6-4122-b6e9-e96f410809ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:16.213715 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61843b47-4db6-4122-b6e9-e96f410809ef admin admin] PATH_INFO: `/v3/users/d304395e2a0a49ceae4c4d9166ca92e2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:16.215493 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61843b47-4db6-4122-b6e9-e96f410809ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=d304395e2a0a49ceae4c4d9166ca92e2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:16.218209 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61843b47-4db6-4122-b6e9-e96f410809ef admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:16.221013 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1217/2446] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:36:16 2022] PUT /identity/v3/projects/70a45f947ad544e188c14845d6b672a5/users/a2e187292e594683a52502b0fda86794/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:36:16.232199 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1abd9c-6ef5-4137-867e-5e3ece190849 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:16.232713 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1abd9c-6ef5-4137-867e-5e3ece190849 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:16.233148 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1abd9c-6ef5-4137-867e-5e3ece190849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:16.267420 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-1c1abd9c-6ef5-4137-867e-5e3ece190849 None None] MFA Rules not processed for user `a2e187292e594683a52502b0fda86794`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:16.268933 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1c1abd9c-6ef5-4137-867e-5e3ece190849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:16.339450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1218/2447] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:16 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:36:16.381251 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1abd9c-6ef5-4137-867e-5e3ece190849 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:16.382745 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1c1abd9c-6ef5-4137-867e-5e3ece190849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:16.445619 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1abd9c-6ef5-4137-867e-5e3ece190849 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:16.449608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434e914-8c50-4d40-ad4a-34d3fa1f79f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:16.450226 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434e914-8c50-4d40-ad4a-34d3fa1f79f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:16.450507 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434e914-8c50-4d40-ad4a-34d3fa1f79f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:16.450819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434e914-8c50-4d40-ad4a-34d3fa1f79f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:16.462136 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f434e914-8c50-4d40-ad4a-34d3fa1f79f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:16.462855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-61843b47-4db6-4122-b6e9-e96f410809ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd304395e2a0a49ceae4c4d9166ca92e2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:16.465304 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f434e914-8c50-4d40-ad4a-34d3fa1f79f4 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:16.470455 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-61843b47-4db6-4122-b6e9-e96f410809ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd304395e2a0a49ceae4c4d9166ca92e2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:16.474156 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1219/2448] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:36:16 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:36:16.489435 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434e914-8c50-4d40-ad4a-34d3fa1f79f4 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:16.524340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:16.549488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-61843b47-4db6-4122-b6e9-e96f410809ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd304395e2a0a49ceae4c4d9166ca92e2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:16.549816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-61843b47-4db6-4122-b6e9-e96f410809ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd304395e2a0a49ceae4c4d9166ca92e2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:16.557372 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434e914-8c50-4d40-ad4a-34d3fa1f79f4 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:16.562189 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebb1595-2843-4688-9ebb-53d05dd13569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:16.562893 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb1595-2843-4688-9ebb-53d05dd13569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:16.563279 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb1595-2843-4688-9ebb-53d05dd13569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:16.563455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb1595-2843-4688-9ebb-53d05dd13569 admin admin] PATH_INFO: `/v3/users/a2e187292e594683a52502b0fda86794` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:16.565020 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebb1595-2843-4688-9ebb-53d05dd13569 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2e187292e594683a52502b0fda86794)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:16.566241 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-61843b47-4db6-4122-b6e9-e96f410809ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd304395e2a0a49ceae4c4d9166ca92e2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:16.566519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebb1595-2843-4688-9ebb-53d05dd13569 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:16.575078 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1230/2449] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:36:16 2022] DELETE /identity/v3/users/d304395e2a0a49ceae4c4d9166ca92e2 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:36:16.579786 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240526c-5b7f-410e-904c-899340bba800 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:16.580242 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240526c-5b7f-410e-904c-899340bba800 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:16.580603 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4240526c-5b7f-410e-904c-899340bba800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:16.603683 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4240526c-5b7f-410e-904c-899340bba800 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:16.604963 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4240526c-5b7f-410e-904c-899340bba800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:16.665810 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1231/2450] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:36:16.680664 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4240526c-5b7f-410e-904c-899340bba800 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:16.680664 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4240526c-5b7f-410e-904c-899340bba800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:16.738167 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:16.744283 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4240526c-5b7f-410e-904c-899340bba800 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:16.748183 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0658dad-cc53-43c8-bc2e-ec3b3cc38f28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:16.748806 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0658dad-cc53-43c8-bc2e-ec3b3cc38f28 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:16.749071 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0658dad-cc53-43c8-bc2e-ec3b3cc38f28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:16.749344 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0658dad-cc53-43c8-bc2e-ec3b3cc38f28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:16.759285 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0658dad-cc53-43c8-bc2e-ec3b3cc38f28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:16.760915 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0658dad-cc53-43c8-bc2e-ec3b3cc38f28 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:16.768630 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6ebb1595-2843-4688-9ebb-53d05dd13569 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2e187292e594683a52502b0fda86794'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:16.771089 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1232/2451] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:36:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:36:16.771739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0658dad-cc53-43c8-bc2e-ec3b3cc38f28 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:16.773218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d0658dad-cc53-43c8-bc2e-ec3b3cc38f28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:16.785776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0658dad-cc53-43c8-bc2e-ec3b3cc38f28 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:16.790633 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52ef85-100e-48e5-aaeb-78abaf4c94af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:16.791407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52ef85-100e-48e5-aaeb-78abaf4c94af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:16.791697 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52ef85-100e-48e5-aaeb-78abaf4c94af admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:16.791972 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52ef85-100e-48e5-aaeb-78abaf4c94af admin admin] PATH_INFO: `/v3/projects/88c9d4087b544e90adcd37236809fa2e` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:16.793605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d52ef85-100e-48e5-aaeb-78abaf4c94af admin admin] RBAC: Authorizing `identity:delete_project(project_id=88c9d4087b544e90adcd37236809fa2e)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:16.795721 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d52ef85-100e-48e5-aaeb-78abaf4c94af admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:16.875563 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6ebb1595-2843-4688-9ebb-53d05dd13569 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2e187292e594683a52502b0fda86794'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:16.876268 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6ebb1595-2843-4688-9ebb-53d05dd13569 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2e187292e594683a52502b0fda86794'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:16.876984 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6ebb1595-2843-4688-9ebb-53d05dd13569 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2e187292e594683a52502b0fda86794'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:16.890094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-6ebb1595-2843-4688-9ebb-53d05dd13569 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2e187292e594683a52502b0fda86794'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:16.902679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1220/2452] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:36:16 2022] DELETE /identity/v3/users/a2e187292e594683a52502b0fda86794 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:36:16.907941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f305a07-38ec-4ab7-858f-ba2344f26331 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:16.908227 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f305a07-38ec-4ab7-858f-ba2344f26331 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:16.908503 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f305a07-38ec-4ab7-858f-ba2344f26331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:16.929799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9f305a07-38ec-4ab7-858f-ba2344f26331 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:16.931246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9f305a07-38ec-4ab7-858f-ba2344f26331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:16.935588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4d52ef85-100e-48e5-aaeb-78abaf4c94af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88c9d4087b544e90adcd37236809fa2e', 'request_id': 'req-4d52ef85-100e-48e5-aaeb-78abaf4c94af'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:16.990962 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1221/2453] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:36:17.000836 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f305a07-38ec-4ab7-858f-ba2344f26331 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:17.002187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9f305a07-38ec-4ab7-858f-ba2344f26331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:17.034812 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4d52ef85-100e-48e5-aaeb-78abaf4c94af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88c9d4087b544e90adcd37236809fa2e', 'request_id': 'req-4d52ef85-100e-48e5-aaeb-78abaf4c94af'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:17.049116 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4d52ef85-100e-48e5-aaeb-78abaf4c94af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88c9d4087b544e90adcd37236809fa2e', 'request_id': 'req-4d52ef85-100e-48e5-aaeb-78abaf4c94af'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:17.063472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:17.067223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f305a07-38ec-4ab7-858f-ba2344f26331 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:17.070499 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4d52ef85-100e-48e5-aaeb-78abaf4c94af admin admin] The token cache is being invalidate because project 88c9d4087b544e90adcd37236809fa2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:36:17.073901 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4d52ef85-100e-48e5-aaeb-78abaf4c94af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:17.074177 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c4d9c9-d05e-42c6-b76a-0b3db17978c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:17.074292 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4d52ef85-100e-48e5-aaeb-78abaf4c94af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:17.074385 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4d9c9-d05e-42c6-b76a-0b3db17978c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:17.074385 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4d9c9-d05e-42c6-b76a-0b3db17978c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:17.074385 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4d9c9-d05e-42c6-b76a-0b3db17978c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:17.074661 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1233/2454] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:36:16 2022] DELETE /identity/v3/projects/88c9d4087b544e90adcd37236809fa2e => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:36:17.074768 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d1c4d9c9-d05e-42c6-b76a-0b3db17978c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:17.077004 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52ef85-100e-48e5-aaeb-78abaf4c94af admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:17.078384 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4d52ef85-100e-48e5-aaeb-78abaf4c94af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:17.087574 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c4d9c9-d05e-42c6-b76a-0b3db17978c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:17.089017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c4d9c9-d05e-42c6-b76a-0b3db17978c6 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:17.091186 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52ef85-100e-48e5-aaeb-78abaf4c94af admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:17.095995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204591bb-253b-457d-9b92-6bcd50981534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:17.096617 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204591bb-253b-457d-9b92-6bcd50981534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:17.096900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204591bb-253b-457d-9b92-6bcd50981534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:17.097172 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204591bb-253b-457d-9b92-6bcd50981534 admin admin] PATH_INFO: `/v3/projects/f81d6a5715cf497fbda2b68647c0a5a2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:17.098271 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204591bb-253b-457d-9b92-6bcd50981534 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f81d6a5715cf497fbda2b68647c0a5a2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:17.100127 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1222/2455] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:36:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:36:17.100769 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204591bb-253b-457d-9b92-6bcd50981534 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:17.101463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f42bf8-c64b-4f82-b099-e3146875e4e0 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:17.101797 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f42bf8-c64b-4f82-b099-e3146875e4e0 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:17.102079 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f42bf8-c64b-4f82-b099-e3146875e4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:17.124759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e3f42bf8-c64b-4f82-b099-e3146875e4e0 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:17.126448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e3f42bf8-c64b-4f82-b099-e3146875e4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:17.134654 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1223/2456] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:36:17.142861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676deb31-6fb0-40e9-8da0-399f85612fc4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:17.143134 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676deb31-6fb0-40e9-8da0-399f85612fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:17.146452 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676deb31-6fb0-40e9-8da0-399f85612fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:17.164511 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-676deb31-6fb0-40e9-8da0-399f85612fc4 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:17.165798 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-676deb31-6fb0-40e9-8da0-399f85612fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:17.173879 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1224/2457] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:36:17.178531 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676deb31-6fb0-40e9-8da0-399f85612fc4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:17.188443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676deb31-6fb0-40e9-8da0-399f85612fc4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:17.193155 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678addc-3cfe-4a54-803b-0b48aaf303f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:17.193754 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678addc-3cfe-4a54-803b-0b48aaf303f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:17.194031 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678addc-3cfe-4a54-803b-0b48aaf303f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:17.194308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678addc-3cfe-4a54-803b-0b48aaf303f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:17.194985 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1678addc-3cfe-4a54-803b-0b48aaf303f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:36:17.195293 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1678addc-3cfe-4a54-803b-0b48aaf303f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:17.196876 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1678addc-3cfe-4a54-803b-0b48aaf303f7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:17.208491 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1225/2458] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:36:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:36:17.212682 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678addc-3cfe-4a54-803b-0b48aaf303f7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:17.221882 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678addc-3cfe-4a54-803b-0b48aaf303f7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:17.227062 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0eb89-6bf4-421e-81e2-8a2daa3388c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:17.227919 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0eb89-6bf4-421e-81e2-8a2daa3388c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:17.228200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0eb89-6bf4-421e-81e2-8a2daa3388c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:17.228476 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0eb89-6bf4-421e-81e2-8a2daa3388c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:17.229090 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c0eb89-6bf4-421e-81e2-8a2daa3388c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:17.230972 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c0eb89-6bf4-421e-81e2-8a2daa3388c7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:17.246771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-204591bb-253b-457d-9b92-6bcd50981534 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f81d6a5715cf497fbda2b68647c0a5a2', 'request_id': 'req-204591bb-253b-457d-9b92-6bcd50981534'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:17.379829 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1226/2459] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:36:17 2022] POST /identity/v3/projects => generated 397 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:36:17.384167 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0eb89-6bf4-421e-81e2-8a2daa3388c7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:17.435260 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-204591bb-253b-457d-9b92-6bcd50981534 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f81d6a5715cf497fbda2b68647c0a5a2', 'request_id': 'req-204591bb-253b-457d-9b92-6bcd50981534'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:17.441783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-204591bb-253b-457d-9b92-6bcd50981534 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f81d6a5715cf497fbda2b68647c0a5a2', 'request_id': 'req-204591bb-253b-457d-9b92-6bcd50981534'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:17.448665 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0eb89-6bf4-421e-81e2-8a2daa3388c7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:17.453766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446cf717-c69d-4a8d-a37e-64f6a953f2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:17.454011 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-204591bb-253b-457d-9b92-6bcd50981534 admin admin] The token cache is being invalidate because project f81d6a5715cf497fbda2b68647c0a5a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:36:17.454365 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-204591bb-253b-457d-9b92-6bcd50981534 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:17.454479 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446cf717-c69d-4a8d-a37e-64f6a953f2f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:17.454669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446cf717-c69d-4a8d-a37e-64f6a953f2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:17.454982 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446cf717-c69d-4a8d-a37e-64f6a953f2f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:17.455094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-204591bb-253b-457d-9b92-6bcd50981534 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:17.455616 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446cf717-c69d-4a8d-a37e-64f6a953f2f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:17.456501 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1234/2460] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:36:17 2022] DELETE /identity/v3/projects/f81d6a5715cf497fbda2b68647c0a5a2 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:36:17.459913 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446cf717-c69d-4a8d-a37e-64f6a953f2f4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:17.552823 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204591bb-253b-457d-9b92-6bcd50981534 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:17.554278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1227/2461] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:36:17 2022] POST /identity/v3/users => generated 299 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:36:17.555721 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446cf717-c69d-4a8d-a37e-64f6a953f2f4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:17.557723 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-446cf717-c69d-4a8d-a37e-64f6a953f2f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:17.558392 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-204591bb-253b-457d-9b92-6bcd50981534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:17.587492 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446cf717-c69d-4a8d-a37e-64f6a953f2f4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:17.593565 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08916ef7-2ae1-45a1-9120-6fd3c9567b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:17.594117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08916ef7-2ae1-45a1-9120-6fd3c9567b91 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:17.594555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08916ef7-2ae1-45a1-9120-6fd3c9567b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:17.594869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08916ef7-2ae1-45a1-9120-6fd3c9567b91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:17.600047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08916ef7-2ae1-45a1-9120-6fd3c9567b91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:17.600047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08916ef7-2ae1-45a1-9120-6fd3c9567b91 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:17.609680 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1228/2462] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:36:17.613405 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08916ef7-2ae1-45a1-9120-6fd3c9567b91 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:17.626478 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08916ef7-2ae1-45a1-9120-6fd3c9567b91 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:17.640221 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5987bb-f897-4b35-9afc-a929f224a274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:17.641068 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5987bb-f897-4b35-9afc-a929f224a274 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:17.641428 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5987bb-f897-4b35-9afc-a929f224a274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:17.641771 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5987bb-f897-4b35-9afc-a929f224a274 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:17.642641 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5987bb-f897-4b35-9afc-a929f224a274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:17.644338 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5987bb-f897-4b35-9afc-a929f224a274 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:17.662429 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:17.668605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1229/2463] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:36:17.668753 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204591bb-253b-457d-9b92-6bcd50981534 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:17.668854 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5987bb-f897-4b35-9afc-a929f224a274 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:17.672016 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c74ca8-f7d2-485d-a85c-e2ce93d7ba00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:17.672662 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c74ca8-f7d2-485d-a85c-e2ce93d7ba00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:17.672946 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c74ca8-f7d2-485d-a85c-e2ce93d7ba00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:17.673286 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c74ca8-f7d2-485d-a85c-e2ce93d7ba00 admin admin] PATH_INFO: `/v3/projects/70a45f947ad544e188c14845d6b672a5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:17.674999 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c74ca8-f7d2-485d-a85c-e2ce93d7ba00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70a45f947ad544e188c14845d6b672a5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:17.676416 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c74ca8-f7d2-485d-a85c-e2ce93d7ba00 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:17.678393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5987bb-f897-4b35-9afc-a929f224a274 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:17.683756 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9930d4d-2294-42fb-bf1d-41fde6cd77a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:17.684421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9930d4d-2294-42fb-bf1d-41fde6cd77a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:17.684889 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9930d4d-2294-42fb-bf1d-41fde6cd77a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:17.685104 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9930d4d-2294-42fb-bf1d-41fde6cd77a4 admin admin] PATH_INFO: `/v3/projects/d3381196cbb0443c99aa89256f2cdee6/users/3bc34babbc8042e3b5592aeda9e0f6bc/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:17.705456 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9930d4d-2294-42fb-bf1d-41fde6cd77a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3381196cbb0443c99aa89256f2cdee6, user_id=3bc34babbc8042e3b5592aeda9e0f6bc, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:17.710939 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9930d4d-2294-42fb-bf1d-41fde6cd77a4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:17.823188 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1230/2464] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:36:17 2022] PUT /identity/v3/projects/d3381196cbb0443c99aa89256f2cdee6/users/3bc34babbc8042e3b5592aeda9e0f6bc/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:36:17.829996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-24c74ca8-f7d2-485d-a85c-e2ce93d7ba00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70a45f947ad544e188c14845d6b672a5', 'request_id': 'req-24c74ca8-f7d2-485d-a85c-e2ce93d7ba00'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:17.833844 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cab896-c91b-4715-ac4e-e0e191edd82e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:17.834131 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cab896-c91b-4715-ac4e-e0e191edd82e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:17.834406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cab896-c91b-4715-ac4e-e0e191edd82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:17.861673 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-67cab896-c91b-4715-ac4e-e0e191edd82e None None] MFA Rules not processed for user `3bc34babbc8042e3b5592aeda9e0f6bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:17.862932 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-67cab896-c91b-4715-ac4e-e0e191edd82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:17.897014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:17.929093 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1231/2465] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:17 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:36:17.942128 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-24c74ca8-f7d2-485d-a85c-e2ce93d7ba00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70a45f947ad544e188c14845d6b672a5', 'request_id': 'req-24c74ca8-f7d2-485d-a85c-e2ce93d7ba00'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:17.948180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-24c74ca8-f7d2-485d-a85c-e2ce93d7ba00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70a45f947ad544e188c14845d6b672a5', 'request_id': 'req-24c74ca8-f7d2-485d-a85c-e2ce93d7ba00'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:17.960618 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cab896-c91b-4715-ac4e-e0e191edd82e None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:17.961748 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-24c74ca8-f7d2-485d-a85c-e2ce93d7ba00 admin admin] The token cache is being invalidate because project 70a45f947ad544e188c14845d6b672a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:36:17.962040 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-24c74ca8-f7d2-485d-a85c-e2ce93d7ba00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:17.962153 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-67cab896-c91b-4715-ac4e-e0e191edd82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:17.962693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-24c74ca8-f7d2-485d-a85c-e2ce93d7ba00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:17.964481 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1235/2466] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:36:17 2022] DELETE /identity/v3/projects/70a45f947ad544e188c14845d6b672a5 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:36:17.968885 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bcdfee-61ad-46c9-a540-f4f2b083695c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:17.969237 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bcdfee-61ad-46c9-a540-f4f2b083695c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:17.969580 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bcdfee-61ad-46c9-a540-f4f2b083695c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:17.992089 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f2bcdfee-61ad-46c9-a540-f4f2b083695c None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:17.993357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f2bcdfee-61ad-46c9-a540-f4f2b083695c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:18.031466 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cab896-c91b-4715-ac4e-e0e191edd82e None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:18.035147 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c305eb7-2baa-4b33-9c40-86586810b7e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:18.035757 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c305eb7-2baa-4b33-9c40-86586810b7e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:18.036032 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c305eb7-2baa-4b33-9c40-86586810b7e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:18.036339 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c305eb7-2baa-4b33-9c40-86586810b7e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:18.038059 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1c305eb7-2baa-4b33-9c40-86586810b7e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:18.049670 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c305eb7-2baa-4b33-9c40-86586810b7e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:18.051160 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c305eb7-2baa-4b33-9c40-86586810b7e1 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:18.056522 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1236/2467] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:36:18.057550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bcdfee-61ad-46c9-a540-f4f2b083695c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:18.058829 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f2bcdfee-61ad-46c9-a540-f4f2b083695c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:18.061636 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1232/2468] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:36:17 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:36:18.064331 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8ec76-67dc-4ecb-9f16-bd8022894465 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:18.070331 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8ec76-67dc-4ecb-9f16-bd8022894465 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:18.070331 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8ec76-67dc-4ecb-9f16-bd8022894465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:18.074746 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bcdfee-61ad-46c9-a540-f4f2b083695c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:18.078913 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a084de0e-b2b7-4525-8171-1b8e22363ddb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:18.079736 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084de0e-b2b7-4525-8171-1b8e22363ddb service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:18.080581 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084de0e-b2b7-4525-8171-1b8e22363ddb service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:18.081161 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a084de0e-b2b7-4525-8171-1b8e22363ddb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:18.087539 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a084de0e-b2b7-4525-8171-1b8e22363ddb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:18.087647 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b1d8ec76-67dc-4ecb-9f16-bd8022894465 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:18.093532 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b1d8ec76-67dc-4ecb-9f16-bd8022894465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:18.097172 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1233/2469] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:36:18.101538 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8ec76-67dc-4ecb-9f16-bd8022894465 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:18.111388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8ec76-67dc-4ecb-9f16-bd8022894465 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:18.116893 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28589503-5f5f-4209-bdd5-1be3a66f3773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:18.125733 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28589503-5f5f-4209-bdd5-1be3a66f3773 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:18.125733 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28589503-5f5f-4209-bdd5-1be3a66f3773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:18.125733 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28589503-5f5f-4209-bdd5-1be3a66f3773 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:18.125733 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28589503-5f5f-4209-bdd5-1be3a66f3773 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:36:18.125733 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28589503-5f5f-4209-bdd5-1be3a66f3773 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:18.125733 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28589503-5f5f-4209-bdd5-1be3a66f3773 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:18.135224 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1234/2470] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:36:18.139668 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28589503-5f5f-4209-bdd5-1be3a66f3773 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:18.152148 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28589503-5f5f-4209-bdd5-1be3a66f3773 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:18.154726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a084de0e-b2b7-4525-8171-1b8e22363ddb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:18.160408 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a084de0e-b2b7-4525-8171-1b8e22363ddb service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:18.160514 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21983e26-f5d3-4863-841d-b980c12cf3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:18.160514 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21983e26-f5d3-4863-841d-b980c12cf3af admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:18.160514 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21983e26-f5d3-4863-841d-b980c12cf3af admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:18.160514 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21983e26-f5d3-4863-841d-b980c12cf3af admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:18.160872 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21983e26-f5d3-4863-841d-b980c12cf3af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:18.162480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21983e26-f5d3-4863-841d-b980c12cf3af admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:18.166519 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1237/2471] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:36:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:36:18.252938 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1235/2472] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:36:18 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:36:18.254121 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a084de0e-b2b7-4525-8171-1b8e22363ddb service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:18.265093 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a084de0e-b2b7-4525-8171-1b8e22363ddb service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:18.286210 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:18.324321 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4512cd-4f5d-4961-b92d-bf907942a5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:18.324975 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4512cd-4f5d-4961-b92d-bf907942a5ba admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:18.325257 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4512cd-4f5d-4961-b92d-bf907942a5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:18.325531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4512cd-4f5d-4961-b92d-bf907942a5ba admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:18.326209 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4512cd-4f5d-4961-b92d-bf907942a5ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:18.328184 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4512cd-4f5d-4961-b92d-bf907942a5ba admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:18.435906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1238/2473] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:36:18 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:36:18.439680 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21983e26-f5d3-4863-841d-b980c12cf3af admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:18.450017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21983e26-f5d3-4863-841d-b980c12cf3af admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:18.454752 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1a9ea-6f9e-4e07-9711-5924e5056a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:18.455362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1a9ea-6f9e-4e07-9711-5924e5056a45 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:18.457058 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1a9ea-6f9e-4e07-9711-5924e5056a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:18.457058 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c1a9ea-6f9e-4e07-9711-5924e5056a45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:18.457058 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c1a9ea-6f9e-4e07-9711-5924e5056a45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:18.458152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c1a9ea-6f9e-4e07-9711-5924e5056a45 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:18.467986 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1236/2474] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:36:18.471926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4512cd-4f5d-4961-b92d-bf907942a5ba admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:18.482254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4512cd-4f5d-4961-b92d-bf907942a5ba admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:18.486865 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e41531-c34f-4fa1-949e-837a90e2646a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:18.487461 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e41531-c34f-4fa1-949e-837a90e2646a admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:18.487744 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e41531-c34f-4fa1-949e-837a90e2646a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:18.488021 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e41531-c34f-4fa1-949e-837a90e2646a admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:18.488665 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e41531-c34f-4fa1-949e-837a90e2646a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:18.490268 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e41531-c34f-4fa1-949e-837a90e2646a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:18.500960 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1239/2475] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:36:18.504969 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1a9ea-6f9e-4e07-9711-5924e5056a45 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:18.516475 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c1a9ea-6f9e-4e07-9711-5924e5056a45 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:18.521738 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459733e9-6000-4ad0-b305-2d823b3bbe13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:18.522696 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459733e9-6000-4ad0-b305-2d823b3bbe13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:18.523005 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459733e9-6000-4ad0-b305-2d823b3bbe13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:18.523487 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459733e9-6000-4ad0-b305-2d823b3bbe13 admin admin] PATH_INFO: `/v3/projects/4648e430b3354141809e7d7e8c0f10bb/users/266f87d9ee3343308d951dcf425fecd9/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:18.540512 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459733e9-6000-4ad0-b305-2d823b3bbe13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4648e430b3354141809e7d7e8c0f10bb, user_id=266f87d9ee3343308d951dcf425fecd9, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:18.543323 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459733e9-6000-4ad0-b305-2d823b3bbe13 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:18.613041 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1237/2476] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:36:18 2022] PUT /identity/v3/projects/4648e430b3354141809e7d7e8c0f10bb/users/266f87d9ee3343308d951dcf425fecd9/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:36:18.625575 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e74b0-cdcc-4389-98d9-9ad5fc153bf4 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:18.626086 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e74b0-cdcc-4389-98d9-9ad5fc153bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:18.626086 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e74b0-cdcc-4389-98d9-9ad5fc153bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:18.653937 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a70e74b0-cdcc-4389-98d9-9ad5fc153bf4 None None] MFA Rules not processed for user `266f87d9ee3343308d951dcf425fecd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:18.655221 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a70e74b0-cdcc-4389-98d9-9ad5fc153bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:18.717219 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1240/2477] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:36:18.721754 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459733e9-6000-4ad0-b305-2d823b3bbe13 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:18.784500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459733e9-6000-4ad0-b305-2d823b3bbe13 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:18.789676 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3ffca-e986-424e-aabd-5005caaf9124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:18.790283 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3ffca-e986-424e-aabd-5005caaf9124 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:18.790564 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3ffca-e986-424e-aabd-5005caaf9124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:18.790878 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3ffca-e986-424e-aabd-5005caaf9124 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:18.791509 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3ffca-e986-424e-aabd-5005caaf9124 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:18.793432 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3ffca-e986-424e-aabd-5005caaf9124 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:18.798666 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:18.906568 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1238/2478] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:36:18 2022] POST /identity/v3/projects => generated 385 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:36:18.910602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e74b0-cdcc-4389-98d9-9ad5fc153bf4 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:18.922859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e74b0-cdcc-4389-98d9-9ad5fc153bf4 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:18.989899 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2b761-f552-4199-85ad-ad753e6a46de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:18.991082 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2b761-f552-4199-85ad-ad753e6a46de admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:18.991368 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2b761-f552-4199-85ad-ad753e6a46de admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:18.991649 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2b761-f552-4199-85ad-ad753e6a46de admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:18.992299 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2b761-f552-4199-85ad-ad753e6a46de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:18.994220 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2b761-f552-4199-85ad-ad753e6a46de admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:19.069465 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1241/2479] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:36:18 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:36:19.073440 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3ffca-e986-424e-aabd-5005caaf9124 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:19.083800 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3ffca-e986-424e-aabd-5005caaf9124 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:19.090546 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a537c-9317-43a9-9aab-f390cba99138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:19.091350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a537c-9317-43a9-9aab-f390cba99138 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:19.091639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a537c-9317-43a9-9aab-f390cba99138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:19.092165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842a537c-9317-43a9-9aab-f390cba99138 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:19.093070 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842a537c-9317-43a9-9aab-f390cba99138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:19.096006 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842a537c-9317-43a9-9aab-f390cba99138 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:19.108366 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1239/2480] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:36:19.112559 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2b761-f552-4199-85ad-ad753e6a46de admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:19.122514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2b761-f552-4199-85ad-ad753e6a46de admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:19.127415 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21305ba7-db50-4f0c-9f26-93596a5a5ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:19.128094 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21305ba7-db50-4f0c-9f26-93596a5a5ccd admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:19.128408 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21305ba7-db50-4f0c-9f26-93596a5a5ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:19.128681 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21305ba7-db50-4f0c-9f26-93596a5a5ccd admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:19.129321 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21305ba7-db50-4f0c-9f26-93596a5a5ccd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:19.131024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21305ba7-db50-4f0c-9f26-93596a5a5ccd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:19.147010 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1242/2481] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:36:19.148883 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a537c-9317-43a9-9aab-f390cba99138 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:19.159065 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842a537c-9317-43a9-9aab-f390cba99138 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:19.163598 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d87f68-29ce-4481-a3ad-5920f86f8f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:19.164246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d87f68-29ce-4481-a3ad-5920f86f8f02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:19.164526 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d87f68-29ce-4481-a3ad-5920f86f8f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:19.164796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d87f68-29ce-4481-a3ad-5920f86f8f02 admin admin] PATH_INFO: `/v3/projects/2c0d2c395a864f3d957e30984f9ed76e/users/8dd3a3efe7274c5bad82cf3b0bfd5167/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:19.178831 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d87f68-29ce-4481-a3ad-5920f86f8f02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c0d2c395a864f3d957e30984f9ed76e, user_id=8dd3a3efe7274c5bad82cf3b0bfd5167, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:19.181488 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d87f68-29ce-4481-a3ad-5920f86f8f02 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:19.280907 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1240/2482] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:36:19 2022] PUT /identity/v3/projects/2c0d2c395a864f3d957e30984f9ed76e/users/8dd3a3efe7274c5bad82cf3b0bfd5167/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:36:19.291545 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e16a22-1350-41ec-96ee-be1c145bf85f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:19.291840 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e16a22-1350-41ec-96ee-be1c145bf85f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:19.292119 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e16a22-1350-41ec-96ee-be1c145bf85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:19.313201 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:19.321041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-34e16a22-1350-41ec-96ee-be1c145bf85f None None] MFA Rules not processed for user `8dd3a3efe7274c5bad82cf3b0bfd5167`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:19.322272 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-34e16a22-1350-41ec-96ee-be1c145bf85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:19.385677 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1243/2483] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:36:19.390161 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d87f68-29ce-4481-a3ad-5920f86f8f02 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:19.452477 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d87f68-29ce-4481-a3ad-5920f86f8f02 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:19.457161 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9258cb-9c34-46ed-8cb4-9be89f400eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:19.457766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9258cb-9c34-46ed-8cb4-9be89f400eb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:19.458048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9258cb-9c34-46ed-8cb4-9be89f400eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:19.458326 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9258cb-9c34-46ed-8cb4-9be89f400eb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:19.458992 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9258cb-9c34-46ed-8cb4-9be89f400eb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:19.461495 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9258cb-9c34-46ed-8cb4-9be89f400eb4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:19.602342 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1241/2484] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:36:19 2022] POST /identity/v3/projects => generated 385 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:36:19.606585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e16a22-1350-41ec-96ee-be1c145bf85f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:19.617529 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e16a22-1350-41ec-96ee-be1c145bf85f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:19.674418 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae470182-19a7-44cc-80ae-9d51a1451561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:19.675072 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae470182-19a7-44cc-80ae-9d51a1451561 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:19.675336 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae470182-19a7-44cc-80ae-9d51a1451561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:19.675611 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae470182-19a7-44cc-80ae-9d51a1451561 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:19.676247 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae470182-19a7-44cc-80ae-9d51a1451561 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:19.678146 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae470182-19a7-44cc-80ae-9d51a1451561 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:19.780001 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1244/2485] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:36:19 2022] POST /identity/v3/users => generated 299 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:36:19.782677 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9258cb-9c34-46ed-8cb4-9be89f400eb4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:19.793475 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9258cb-9c34-46ed-8cb4-9be89f400eb4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:19.795424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae470182-19a7-44cc-80ae-9d51a1451561 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:19.800165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e381d-e5c9-4aaa-b105-923cb43855b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:19.800165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e381d-e5c9-4aaa-b105-923cb43855b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:19.800165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e381d-e5c9-4aaa-b105-923cb43855b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:19.800165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e381d-e5c9-4aaa-b105-923cb43855b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:19.800560 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5e381d-e5c9-4aaa-b105-923cb43855b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:19.801902 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5e381d-e5c9-4aaa-b105-923cb43855b6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:19.813644 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1242/2486] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:36:19.818096 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e381d-e5c9-4aaa-b105-923cb43855b6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:19.829062 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e381d-e5c9-4aaa-b105-923cb43855b6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:19.833770 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2834efb-03f0-4963-8bad-a8f40802d516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:19.834488 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2834efb-03f0-4963-8bad-a8f40802d516 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:19.834803 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2834efb-03f0-4963-8bad-a8f40802d516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:19.835078 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2834efb-03f0-4963-8bad-a8f40802d516 admin admin] PATH_INFO: `/v3/projects/c7aa63907fcf4f2a8d26261154ddb930/users/bc7f12d0bf1a47ba89bdba2d338e894b/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:19.859779 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae470182-19a7-44cc-80ae-9d51a1451561 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:19.860108 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2834efb-03f0-4963-8bad-a8f40802d516 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7aa63907fcf4f2a8d26261154ddb930, user_id=bc7f12d0bf1a47ba89bdba2d338e894b, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:19.860213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2b3c8-058f-4a1c-8bdf-bb656996ba38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:19.860213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2b3c8-058f-4a1c-8bdf-bb656996ba38 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:19.860213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2b3c8-058f-4a1c-8bdf-bb656996ba38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:19.860498 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2834efb-03f0-4963-8bad-a8f40802d516 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:19.860588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2b3c8-058f-4a1c-8bdf-bb656996ba38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:19.860588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-10e2b3c8-058f-4a1c-8bdf-bb656996ba38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:19.904983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:19.923714 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e2b3c8-058f-4a1c-8bdf-bb656996ba38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:19.925135 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e2b3c8-058f-4a1c-8bdf-bb656996ba38 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:19.935390 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1245/2487] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:36:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:36:19.958542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1243/2488] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:36:19 2022] PUT /identity/v3/projects/c7aa63907fcf4f2a8d26261154ddb930/users/bc7f12d0bf1a47ba89bdba2d338e894b/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:36:19.962471 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2b3c8-058f-4a1c-8bdf-bb656996ba38 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:19.972937 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2b3c8-058f-4a1c-8bdf-bb656996ba38 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:20.029697 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f8b34-cd7d-4448-8a08-eb9356145cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:20.030341 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f8b34-cd7d-4448-8a08-eb9356145cdc admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:20.030650 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f8b34-cd7d-4448-8a08-eb9356145cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:20.030976 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f8b34-cd7d-4448-8a08-eb9356145cdc admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:20.031610 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f8b34-cd7d-4448-8a08-eb9356145cdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:20.033236 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f8b34-cd7d-4448-8a08-eb9356145cdc admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:20.043660 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1246/2489] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:36:19 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:36:20.048558 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2834efb-03f0-4963-8bad-a8f40802d516 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:20.059328 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2834efb-03f0-4963-8bad-a8f40802d516 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:20.063990 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c94a9-6575-416a-a302-d9896e48019f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:20.064646 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c94a9-6575-416a-a302-d9896e48019f admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:20.064923 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c94a9-6575-416a-a302-d9896e48019f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:20.065223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1c94a9-6575-416a-a302-d9896e48019f admin admin] PATH_INFO: `/v3/projects/c7aa63907fcf4f2a8d26261154ddb930/users/bc7f12d0bf1a47ba89bdba2d338e894b/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:20.067170 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c94a9-6575-416a-a302-d9896e48019f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7aa63907fcf4f2a8d26261154ddb930, user_id=bc7f12d0bf1a47ba89bdba2d338e894b, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:20.072861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1c94a9-6575-416a-a302-d9896e48019f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:20.081443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1244/2490] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:36:20 2022] PUT /identity/v3/projects/c7aa63907fcf4f2a8d26261154ddb930/users/bc7f12d0bf1a47ba89bdba2d338e894b/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:36:20.093694 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb0fa0-b20a-4ac5-84b1-f184a98917ae None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:20.097759 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb0fa0-b20a-4ac5-84b1-f184a98917ae None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:20.097759 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb0fa0-b20a-4ac5-84b1-f184a98917ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:20.126142 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-fdfb0fa0-b20a-4ac5-84b1-f184a98917ae None None] MFA Rules not processed for user `bc7f12d0bf1a47ba89bdba2d338e894b`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:20.127387 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fdfb0fa0-b20a-4ac5-84b1-f184a98917ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:20.200743 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1247/2491] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:20 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:36:20.213552 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c94a9-6575-416a-a302-d9896e48019f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:20.266723 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:20.270432 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1c94a9-6575-416a-a302-d9896e48019f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:20.274646 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5292d87a-1d4d-4a24-8ae5-40a07fed55b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:20.275294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292d87a-1d4d-4a24-8ae5-40a07fed55b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:20.275602 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292d87a-1d4d-4a24-8ae5-40a07fed55b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:20.275883 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5292d87a-1d4d-4a24-8ae5-40a07fed55b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:20.332780 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5292d87a-1d4d-4a24-8ae5-40a07fed55b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:20.334208 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5292d87a-1d4d-4a24-8ae5-40a07fed55b4 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:20.343792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1245/2492] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:36:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:36:23.331750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb0fa0-b20a-4ac5-84b1-f184a98917ae None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:23.341210 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb0fa0-b20a-4ac5-84b1-f184a98917ae None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:23.344782 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6551bd-c0dd-4d44-89a3-481d0d42854b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:23.345399 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6551bd-c0dd-4d44-89a3-481d0d42854b service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:23.345678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6551bd-c0dd-4d44-89a3-481d0d42854b service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:23.345945 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6551bd-c0dd-4d44-89a3-481d0d42854b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:23.401530 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6551bd-c0dd-4d44-89a3-481d0d42854b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:23.402913 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6551bd-c0dd-4d44-89a3-481d0d42854b service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:23.407548 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:23.411962 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1248/2493] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:36:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:36:26.529633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5292d87a-1d4d-4a24-8ae5-40a07fed55b4 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:26.538115 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5292d87a-1d4d-4a24-8ae5-40a07fed55b4 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:26.541668 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8dbaa4-9a1b-483b-9483-6b54d72ff5c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:26.542263 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8dbaa4-9a1b-483b-9483-6b54d72ff5c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:26.542540 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8dbaa4-9a1b-483b-9483-6b54d72ff5c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:26.542852 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8dbaa4-9a1b-483b-9483-6b54d72ff5c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:26.551866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8dbaa4-9a1b-483b-9483-6b54d72ff5c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:26.553337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8dbaa4-9a1b-483b-9483-6b54d72ff5c4 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:26.563301 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1246/2494] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:36:26 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:36:29.003655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6551bd-c0dd-4d44-89a3-481d0d42854b service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:29.005200 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ba6551bd-c0dd-4d44-89a3-481d0d42854b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:29.071224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6551bd-c0dd-4d44-89a3-481d0d42854b service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:29.075925 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add2831f-7e5a-4026-bc10-99ed9562f08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:29.076554 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2831f-7e5a-4026-bc10-99ed9562f08c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:29.076835 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2831f-7e5a-4026-bc10-99ed9562f08c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:29.077115 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2831f-7e5a-4026-bc10-99ed9562f08c admin admin] PATH_INFO: `/v3/users/343bf9821bf84c23a11b8f6d15c49a9c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:29.078256 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add2831f-7e5a-4026-bc10-99ed9562f08c admin admin] RBAC: Authorizing `identity:delete_user(user_id=343bf9821bf84c23a11b8f6d15c49a9c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:29.080260 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add2831f-7e5a-4026-bc10-99ed9562f08c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:29.241921 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8dbaa4-9a1b-483b-9483-6b54d72ff5c4 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:29.243365 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-de8dbaa4-9a1b-483b-9483-6b54d72ff5c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:29.259099 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8dbaa4-9a1b-483b-9483-6b54d72ff5c4 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:29.264242 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01b1ed-8cb0-466a-90a7-32259eca457b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:29.264867 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01b1ed-8cb0-466a-90a7-32259eca457b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:29.265147 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01b1ed-8cb0-466a-90a7-32259eca457b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:29.265422 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01b1ed-8cb0-466a-90a7-32259eca457b admin admin] PATH_INFO: `/v3/users/3bc34babbc8042e3b5592aeda9e0f6bc` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:29.266530 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be01b1ed-8cb0-466a-90a7-32259eca457b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bc34babbc8042e3b5592aeda9e0f6bc)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:29.268473 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be01b1ed-8cb0-466a-90a7-32259eca457b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:29.297147 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-add2831f-7e5a-4026-bc10-99ed9562f08c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '343bf9821bf84c23a11b8f6d15c49a9c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:29.304309 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-add2831f-7e5a-4026-bc10-99ed9562f08c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '343bf9821bf84c23a11b8f6d15c49a9c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:29.378912 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-add2831f-7e5a-4026-bc10-99ed9562f08c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '343bf9821bf84c23a11b8f6d15c49a9c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:29.379488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-add2831f-7e5a-4026-bc10-99ed9562f08c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '343bf9821bf84c23a11b8f6d15c49a9c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:29.392568 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-add2831f-7e5a-4026-bc10-99ed9562f08c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '343bf9821bf84c23a11b8f6d15c49a9c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:29.398468 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1249/2495] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:36:29 2022] DELETE /identity/v3/users/343bf9821bf84c23a11b8f6d15c49a9c => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:36:29.403245 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2993b-1272-466c-b446-dfb970404c80 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:29.403527 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2993b-1272-466c-b446-dfb970404c80 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:29.403792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec2993b-1272-466c-b446-dfb970404c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:29.422923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4ec2993b-1272-466c-b446-dfb970404c80 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:29.424198 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4ec2993b-1272-466c-b446-dfb970404c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:29.459177 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-be01b1ed-8cb0-466a-90a7-32259eca457b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bc34babbc8042e3b5592aeda9e0f6bc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:29.487628 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1250/2496] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:36:29.497338 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec2993b-1272-466c-b446-dfb970404c80 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:29.499582 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4ec2993b-1272-466c-b446-dfb970404c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:29.522077 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:29.528243 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-be01b1ed-8cb0-466a-90a7-32259eca457b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bc34babbc8042e3b5592aeda9e0f6bc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:29.528953 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-be01b1ed-8cb0-466a-90a7-32259eca457b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bc34babbc8042e3b5592aeda9e0f6bc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:29.529636 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-be01b1ed-8cb0-466a-90a7-32259eca457b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bc34babbc8042e3b5592aeda9e0f6bc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:29.543798 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-be01b1ed-8cb0-466a-90a7-32259eca457b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bc34babbc8042e3b5592aeda9e0f6bc'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:29.559098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec2993b-1272-466c-b446-dfb970404c80 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:29.561690 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1247/2497] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:36:29 2022] DELETE /identity/v3/users/3bc34babbc8042e3b5592aeda9e0f6bc => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:36:29.563435 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764347bb-4b4c-498b-a68e-e150e50cf698 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:29.564122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764347bb-4b4c-498b-a68e-e150e50cf698 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:29.564413 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764347bb-4b4c-498b-a68e-e150e50cf698 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:29.564953 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764347bb-4b4c-498b-a68e-e150e50cf698 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:29.565973 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26edfc-7765-4315-934c-47f12fb0bdec None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:29.566305 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26edfc-7765-4315-934c-47f12fb0bdec None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:29.566687 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26edfc-7765-4315-934c-47f12fb0bdec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:29.568168 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-764347bb-4b4c-498b-a68e-e150e50cf698 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:29.584162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764347bb-4b4c-498b-a68e-e150e50cf698 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:29.585594 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764347bb-4b4c-498b-a68e-e150e50cf698 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:29.587043 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ec26edfc-7765-4315-934c-47f12fb0bdec None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:29.588251 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ec26edfc-7765-4315-934c-47f12fb0bdec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:29.597839 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1251/2498] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:36:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:36:29.659112 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1248/2499] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:36:29.669051 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764347bb-4b4c-498b-a68e-e150e50cf698 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:29.670761 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-764347bb-4b4c-498b-a68e-e150e50cf698 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:29.682031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764347bb-4b4c-498b-a68e-e150e50cf698 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:29.733803 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b8016-d77b-4b31-80dd-017acdd2ceb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:29.734437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b8016-d77b-4b31-80dd-017acdd2ceb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:29.734725 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b8016-d77b-4b31-80dd-017acdd2ceb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:29.735025 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b8016-d77b-4b31-80dd-017acdd2ceb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:29.746009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b8016-d77b-4b31-80dd-017acdd2ceb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:29.747466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b8016-d77b-4b31-80dd-017acdd2ceb3 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:29.757274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1252/2500] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:36:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:36:29.828225 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26edfc-7765-4315-934c-47f12fb0bdec None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:29.833375 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:29.839146 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26edfc-7765-4315-934c-47f12fb0bdec None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:29.852231 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6cad1-ba22-4b4c-b7cd-1e9787da1236 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:29.852978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6cad1-ba22-4b4c-b7cd-1e9787da1236 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:29.853198 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6cad1-ba22-4b4c-b7cd-1e9787da1236 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:29.853465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c6cad1-ba22-4b4c-b7cd-1e9787da1236 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:29.855866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-28c6cad1-ba22-4b4c-b7cd-1e9787da1236 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:29.923426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6cad1-ba22-4b4c-b7cd-1e9787da1236 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:29.924862 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c6cad1-ba22-4b4c-b7cd-1e9787da1236 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:29.934910 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1249/2501] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:36:29 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:36:30.138123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b8016-d77b-4b31-80dd-017acdd2ceb3 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:30.140718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5b0b8016-d77b-4b31-80dd-017acdd2ceb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:30.157100 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b8016-d77b-4b31-80dd-017acdd2ceb3 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:30.164766 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7aa375-3f7f-490f-84de-c68357833e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:30.165872 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7aa375-3f7f-490f-84de-c68357833e51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:30.167374 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7aa375-3f7f-490f-84de-c68357833e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:30.167958 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7aa375-3f7f-490f-84de-c68357833e51 admin admin] PATH_INFO: `/v3/projects/0496f560580c4fd59ff85eaab508c538` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:30.170030 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7aa375-3f7f-490f-84de-c68357833e51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0496f560580c4fd59ff85eaab508c538)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:30.172961 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7aa375-3f7f-490f-84de-c68357833e51 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:30.253426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6cad1-ba22-4b4c-b7cd-1e9787da1236 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:30.255042 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-28c6cad1-ba22-4b4c-b7cd-1e9787da1236 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:30.270081 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c6cad1-ba22-4b4c-b7cd-1e9787da1236 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:30.275504 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745278c5-b74d-40a4-8a86-41cafe020875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:30.276397 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745278c5-b74d-40a4-8a86-41cafe020875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:30.282661 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745278c5-b74d-40a4-8a86-41cafe020875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:30.282661 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745278c5-b74d-40a4-8a86-41cafe020875 admin admin] PATH_INFO: `/v3/projects/d3381196cbb0443c99aa89256f2cdee6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:30.282661 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745278c5-b74d-40a4-8a86-41cafe020875 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3381196cbb0443c99aa89256f2cdee6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:30.282661 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745278c5-b74d-40a4-8a86-41cafe020875 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:30.315794 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7f7aa375-3f7f-490f-84de-c68357833e51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0496f560580c4fd59ff85eaab508c538', 'request_id': 'req-7f7aa375-3f7f-490f-84de-c68357833e51'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:30.374727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7f7aa375-3f7f-490f-84de-c68357833e51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0496f560580c4fd59ff85eaab508c538', 'request_id': 'req-7f7aa375-3f7f-490f-84de-c68357833e51'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:30.382519 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7f7aa375-3f7f-490f-84de-c68357833e51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0496f560580c4fd59ff85eaab508c538', 'request_id': 'req-7f7aa375-3f7f-490f-84de-c68357833e51'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:30.394571 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-745278c5-b74d-40a4-8a86-41cafe020875 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3381196cbb0443c99aa89256f2cdee6', 'request_id': 'req-745278c5-b74d-40a4-8a86-41cafe020875'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:30.395997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7f7aa375-3f7f-490f-84de-c68357833e51 admin admin] The token cache is being invalidate because project 0496f560580c4fd59ff85eaab508c538 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:36:30.396384 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7f7aa375-3f7f-490f-84de-c68357833e51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:30.397432 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7f7aa375-3f7f-490f-84de-c68357833e51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:30.399194 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1253/2502] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:36:30 2022] DELETE /identity/v3/projects/0496f560580c4fd59ff85eaab508c538 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:36:30.402012 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-745278c5-b74d-40a4-8a86-41cafe020875 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3381196cbb0443c99aa89256f2cdee6', 'request_id': 'req-745278c5-b74d-40a4-8a86-41cafe020875'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:30.403983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7aa375-3f7f-490f-84de-c68357833e51 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:30.406473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7f7aa375-3f7f-490f-84de-c68357833e51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:30.411571 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-745278c5-b74d-40a4-8a86-41cafe020875 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3381196cbb0443c99aa89256f2cdee6', 'request_id': 'req-745278c5-b74d-40a4-8a86-41cafe020875'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:30.471886 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:30.477386 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-745278c5-b74d-40a4-8a86-41cafe020875 admin admin] The token cache is being invalidate because project d3381196cbb0443c99aa89256f2cdee6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:36:30.477689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-745278c5-b74d-40a4-8a86-41cafe020875 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:30.478633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-745278c5-b74d-40a4-8a86-41cafe020875 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:30.480943 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1250/2503] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:36:30 2022] DELETE /identity/v3/projects/d3381196cbb0443c99aa89256f2cdee6 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:36:30.495269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7aa375-3f7f-490f-84de-c68357833e51 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:30.572400 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18c0b3-408c-4b59-ab97-d0b35735559e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:30.573145 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18c0b3-408c-4b59-ab97-d0b35735559e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:30.573362 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18c0b3-408c-4b59-ab97-d0b35735559e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:30.573634 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18c0b3-408c-4b59-ab97-d0b35735559e admin admin] PATH_INFO: `/v3/users/f48c58f4ed2343d89776cadf3983551d` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:30.574878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18c0b3-408c-4b59-ab97-d0b35735559e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f48c58f4ed2343d89776cadf3983551d)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:30.576852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18c0b3-408c-4b59-ab97-d0b35735559e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:30.798735 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8c18c0b3-408c-4b59-ab97-d0b35735559e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f48c58f4ed2343d89776cadf3983551d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:30.805305 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8c18c0b3-408c-4b59-ab97-d0b35735559e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f48c58f4ed2343d89776cadf3983551d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:30.906271 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8c18c0b3-408c-4b59-ab97-d0b35735559e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f48c58f4ed2343d89776cadf3983551d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:30.907067 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8c18c0b3-408c-4b59-ab97-d0b35735559e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f48c58f4ed2343d89776cadf3983551d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:30.920568 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-8c18c0b3-408c-4b59-ab97-d0b35735559e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f48c58f4ed2343d89776cadf3983551d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:30.928093 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1254/2504] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:36:30 2022] DELETE /identity/v3/users/f48c58f4ed2343d89776cadf3983551d => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:36:31.462133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745278c5-b74d-40a4-8a86-41cafe020875 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:31.463895 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-745278c5-b74d-40a4-8a86-41cafe020875 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:31.476982 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745278c5-b74d-40a4-8a86-41cafe020875 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:31.537501 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363713bf-105c-4476-bdf4-5251f97dbff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:31.538153 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363713bf-105c-4476-bdf4-5251f97dbff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:31.538433 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363713bf-105c-4476-bdf4-5251f97dbff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:31.538765 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363713bf-105c-4476-bdf4-5251f97dbff2 admin admin] PATH_INFO: `/v3/projects/89c677fdd5ed4b0981c9064997f600ba` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:31.539899 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363713bf-105c-4476-bdf4-5251f97dbff2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89c677fdd5ed4b0981c9064997f600ba)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:31.541833 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363713bf-105c-4476-bdf4-5251f97dbff2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:31.567731 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:31.711865 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-363713bf-105c-4476-bdf4-5251f97dbff2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89c677fdd5ed4b0981c9064997f600ba', 'request_id': 'req-363713bf-105c-4476-bdf4-5251f97dbff2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:31.717207 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-363713bf-105c-4476-bdf4-5251f97dbff2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89c677fdd5ed4b0981c9064997f600ba', 'request_id': 'req-363713bf-105c-4476-bdf4-5251f97dbff2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:31.723352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-363713bf-105c-4476-bdf4-5251f97dbff2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89c677fdd5ed4b0981c9064997f600ba', 'request_id': 'req-363713bf-105c-4476-bdf4-5251f97dbff2'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:31.802413 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-363713bf-105c-4476-bdf4-5251f97dbff2 admin admin] The token cache is being invalidate because project 89c677fdd5ed4b0981c9064997f600ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:36:31.802718 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-363713bf-105c-4476-bdf4-5251f97dbff2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:31.803376 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-363713bf-105c-4476-bdf4-5251f97dbff2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:31.804913 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1251/2505] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:36:31 2022] DELETE /identity/v3/projects/89c677fdd5ed4b0981c9064997f600ba => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:36:31.809480 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18c0b3-408c-4b59-ab97-d0b35735559e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:31.810969 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8c18c0b3-408c-4b59-ab97-d0b35735559e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:31.824357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18c0b3-408c-4b59-ab97-d0b35735559e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:31.889019 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9561d38-bd6a-43da-8812-825b2bff45b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:31.889680 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9561d38-bd6a-43da-8812-825b2bff45b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:31.890038 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9561d38-bd6a-43da-8812-825b2bff45b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:31.890315 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9561d38-bd6a-43da-8812-825b2bff45b9 admin admin] PATH_INFO: `/v3/users/fc55015f63c24025973f7416f79bcbff` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:31.891524 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9561d38-bd6a-43da-8812-825b2bff45b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc55015f63c24025973f7416f79bcbff)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:31.893475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9561d38-bd6a-43da-8812-825b2bff45b9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:31.912500 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:32.096276 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d9561d38-bd6a-43da-8812-825b2bff45b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc55015f63c24025973f7416f79bcbff'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:32.109463 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d9561d38-bd6a-43da-8812-825b2bff45b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc55015f63c24025973f7416f79bcbff'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:32.151945 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d9561d38-bd6a-43da-8812-825b2bff45b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc55015f63c24025973f7416f79bcbff'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:32.152565 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d9561d38-bd6a-43da-8812-825b2bff45b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc55015f63c24025973f7416f79bcbff'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:32.164937 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d9561d38-bd6a-43da-8812-825b2bff45b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc55015f63c24025973f7416f79bcbff'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:32.170862 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1255/2506] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:36:31 2022] DELETE /identity/v3/users/fc55015f63c24025973f7416f79bcbff => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:36:32.298106 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363713bf-105c-4476-bdf4-5251f97dbff2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:32.299640 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-363713bf-105c-4476-bdf4-5251f97dbff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:32.313231 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363713bf-105c-4476-bdf4-5251f97dbff2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:32.382783 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dbc303-8ceb-409f-87ab-b0eb77b6ac38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:32.385273 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dbc303-8ceb-409f-87ab-b0eb77b6ac38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:32.385273 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dbc303-8ceb-409f-87ab-b0eb77b6ac38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:32.385273 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dbc303-8ceb-409f-87ab-b0eb77b6ac38 admin admin] PATH_INFO: `/v3/users/266f87d9ee3343308d951dcf425fecd9` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:32.385273 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dbc303-8ceb-409f-87ab-b0eb77b6ac38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=266f87d9ee3343308d951dcf425fecd9)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:32.387307 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dbc303-8ceb-409f-87ab-b0eb77b6ac38 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:32.536143 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a9dbc303-8ceb-409f-87ab-b0eb77b6ac38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '266f87d9ee3343308d951dcf425fecd9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:32.592659 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a9dbc303-8ceb-409f-87ab-b0eb77b6ac38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '266f87d9ee3343308d951dcf425fecd9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:32.593458 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a9dbc303-8ceb-409f-87ab-b0eb77b6ac38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '266f87d9ee3343308d951dcf425fecd9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:32.594184 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a9dbc303-8ceb-409f-87ab-b0eb77b6ac38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '266f87d9ee3343308d951dcf425fecd9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:32.608484 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-a9dbc303-8ceb-409f-87ab-b0eb77b6ac38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '266f87d9ee3343308d951dcf425fecd9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:32.625269 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1252/2507] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:36:32 2022] DELETE /identity/v3/users/266f87d9ee3343308d951dcf425fecd9 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:36:32.628735 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9561d38-bd6a-43da-8812-825b2bff45b9 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:32.630888 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d9561d38-bd6a-43da-8812-825b2bff45b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:32.632269 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54680e9c-a175-4213-9464-909285a81222 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:32.632628 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54680e9c-a175-4213-9464-909285a81222 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:32.636684 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54680e9c-a175-4213-9464-909285a81222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:32.648956 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9561d38-bd6a-43da-8812-825b2bff45b9 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:32.666011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-54680e9c-a175-4213-9464-909285a81222 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:32.668092 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-54680e9c-a175-4213-9464-909285a81222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:32.697563 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:32.748015 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1253/2508] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:36:32.760202 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54680e9c-a175-4213-9464-909285a81222 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:32.762197 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-54680e9c-a175-4213-9464-909285a81222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:32.767228 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c5e15-40b3-4e94-a013-2ce8de0517ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:32.768835 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c5e15-40b3-4e94-a013-2ce8de0517ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:32.769337 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c5e15-40b3-4e94-a013-2ce8de0517ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:32.770005 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c5e15-40b3-4e94-a013-2ce8de0517ad admin admin] PATH_INFO: `/v3/projects/7f0f2ffa2c404e8687438d3f5815e7e7` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:32.771571 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23c5e15-40b3-4e94-a013-2ce8de0517ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f0f2ffa2c404e8687438d3f5815e7e7)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:32.774433 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23c5e15-40b3-4e94-a013-2ce8de0517ad admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:32.837809 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54680e9c-a175-4213-9464-909285a81222 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:32.848009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd6e99-e68d-4316-a26b-dd891352cf29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:32.849184 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd6e99-e68d-4316-a26b-dd891352cf29 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:32.849468 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd6e99-e68d-4316-a26b-dd891352cf29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:32.849739 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd6e99-e68d-4316-a26b-dd891352cf29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:32.862692 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cd6e99-e68d-4316-a26b-dd891352cf29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:32.864613 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cd6e99-e68d-4316-a26b-dd891352cf29 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:32.881149 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1254/2509] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:36:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:36:32.911344 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c23c5e15-40b3-4e94-a013-2ce8de0517ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f0f2ffa2c404e8687438d3f5815e7e7', 'request_id': 'req-c23c5e15-40b3-4e94-a013-2ce8de0517ad'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:33.003632 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c23c5e15-40b3-4e94-a013-2ce8de0517ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f0f2ffa2c404e8687438d3f5815e7e7', 'request_id': 'req-c23c5e15-40b3-4e94-a013-2ce8de0517ad'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:33.010565 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c23c5e15-40b3-4e94-a013-2ce8de0517ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f0f2ffa2c404e8687438d3f5815e7e7', 'request_id': 'req-c23c5e15-40b3-4e94-a013-2ce8de0517ad'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:33.023992 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c23c5e15-40b3-4e94-a013-2ce8de0517ad admin admin] The token cache is being invalidate because project 7f0f2ffa2c404e8687438d3f5815e7e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:36:33.024414 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c23c5e15-40b3-4e94-a013-2ce8de0517ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:33.025337 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c23c5e15-40b3-4e94-a013-2ce8de0517ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:33.027749 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1256/2510] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:36:32 2022] DELETE /identity/v3/projects/7f0f2ffa2c404e8687438d3f5815e7e7 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:36:33.033384 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee94736-2e62-4d4c-b29b-4c4652ed6e0b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:33.033787 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee94736-2e62-4d4c-b29b-4c4652ed6e0b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:33.034591 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee94736-2e62-4d4c-b29b-4c4652ed6e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:33.056542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4ee94736-2e62-4d4c-b29b-4c4652ed6e0b None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:33.058345 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4ee94736-2e62-4d4c-b29b-4c4652ed6e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:33.136994 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1255/2511] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:36:33.145206 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8753e4b-1d67-4830-ac8b-2a4ec4f80ab1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:33.145517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8753e4b-1d67-4830-ac8b-2a4ec4f80ab1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:33.145791 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8753e4b-1d67-4830-ac8b-2a4ec4f80ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:33.168196 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a8753e4b-1d67-4830-ac8b-2a4ec4f80ab1 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:33.169420 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a8753e4b-1d67-4830-ac8b-2a4ec4f80ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:33.178146 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1257/2512] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:36:33.182265 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee94736-2e62-4d4c-b29b-4c4652ed6e0b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:33.197088 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee94736-2e62-4d4c-b29b-4c4652ed6e0b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:33.202581 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc28bb-43fe-47d6-a88c-b21f0cfe0163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:33.203274 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc28bb-43fe-47d6-a88c-b21f0cfe0163 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:33.203552 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc28bb-43fe-47d6-a88c-b21f0cfe0163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:33.203821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc28bb-43fe-47d6-a88c-b21f0cfe0163 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:33.204478 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc28bb-43fe-47d6-a88c-b21f0cfe0163 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:36:33.204914 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc28bb-43fe-47d6-a88c-b21f0cfe0163 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:33.206521 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc28bb-43fe-47d6-a88c-b21f0cfe0163 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:33.224262 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1256/2513] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:36:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:36:33.228420 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8753e4b-1d67-4830-ac8b-2a4ec4f80ab1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:33.239027 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8753e4b-1d67-4830-ac8b-2a4ec4f80ab1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:33.247770 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ca690-b387-44fb-9a10-957060711c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:33.248390 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ca690-b387-44fb-9a10-957060711c94 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:33.248665 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ca690-b387-44fb-9a10-957060711c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:33.248935 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ca690-b387-44fb-9a10-957060711c94 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:33.249595 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ca690-b387-44fb-9a10-957060711c94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:33.251517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ca690-b387-44fb-9a10-957060711c94 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:33.268689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:33.388557 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc28bb-43fe-47d6-a88c-b21f0cfe0163 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:33.390621 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8cdc28bb-43fe-47d6-a88c-b21f0cfe0163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:33.396762 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1258/2514] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:36:33 2022] POST /identity/v3/projects => generated 371 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:36:33.398078 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ca690-b387-44fb-9a10-957060711c94 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:33.409907 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc28bb-43fe-47d6-a88c-b21f0cfe0163 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:33.471866 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ca690-b387-44fb-9a10-957060711c94 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:33.477256 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03bc9e-8036-4224-b75f-3d013204dee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:33.477861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03bc9e-8036-4224-b75f-3d013204dee5 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:33.478143 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03bc9e-8036-4224-b75f-3d013204dee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:33.478418 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03bc9e-8036-4224-b75f-3d013204dee5 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:33.479107 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be03bc9e-8036-4224-b75f-3d013204dee5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:33.481235 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be03bc9e-8036-4224-b75f-3d013204dee5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:33.482494 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb1818-0221-4905-a99a-26962619a4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:33.483164 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb1818-0221-4905-a99a-26962619a4ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:33.483445 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb1818-0221-4905-a99a-26962619a4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:33.483864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb1818-0221-4905-a99a-26962619a4ae admin admin] PATH_INFO: `/v3/projects/4648e430b3354141809e7d7e8c0f10bb` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:33.484994 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eb1818-0221-4905-a99a-26962619a4ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=4648e430b3354141809e7d7e8c0f10bb)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:33.487086 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eb1818-0221-4905-a99a-26962619a4ae admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:33.488689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:33.613030 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1259/2515] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:36:33 2022] POST /identity/v3/users => generated 286 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:36:33.614276 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03bc9e-8036-4224-b75f-3d013204dee5 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:33.625145 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03bc9e-8036-4224-b75f-3d013204dee5 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:33.628507 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-51eb1818-0221-4905-a99a-26962619a4ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4648e430b3354141809e7d7e8c0f10bb', 'request_id': 'req-51eb1818-0221-4905-a99a-26962619a4ae'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:33.634362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-51eb1818-0221-4905-a99a-26962619a4ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4648e430b3354141809e7d7e8c0f10bb', 'request_id': 'req-51eb1818-0221-4905-a99a-26962619a4ae'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:33.642019 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-51eb1818-0221-4905-a99a-26962619a4ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4648e430b3354141809e7d7e8c0f10bb', 'request_id': 'req-51eb1818-0221-4905-a99a-26962619a4ae'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:33.682435 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a4169-7231-4e97-ad1b-f7af4a4a43a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:33.683581 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a4169-7231-4e97-ad1b-f7af4a4a43a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:33.683886 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a4169-7231-4e97-ad1b-f7af4a4a43a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:33.684181 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a4169-7231-4e97-ad1b-f7af4a4a43a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:33.684869 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a4169-7231-4e97-ad1b-f7af4a4a43a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:33.686509 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357a4169-7231-4e97-ad1b-f7af4a4a43a4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:33.696189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1260/2516] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:36:33.700725 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a4169-7231-4e97-ad1b-f7af4a4a43a4 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:33.710039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357a4169-7231-4e97-ad1b-f7af4a4a43a4 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:33.714509 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20131ba3-34bf-47ce-a77b-15897528cbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:33.715556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20131ba3-34bf-47ce-a77b-15897528cbe7 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:33.715556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20131ba3-34bf-47ce-a77b-15897528cbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:33.715757 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20131ba3-34bf-47ce-a77b-15897528cbe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:33.716358 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20131ba3-34bf-47ce-a77b-15897528cbe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:33.717877 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20131ba3-34bf-47ce-a77b-15897528cbe7 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:33.727778 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1261/2517] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 09:36:33.732012 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20131ba3-34bf-47ce-a77b-15897528cbe7 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:33.740036 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-51eb1818-0221-4905-a99a-26962619a4ae admin admin] The token cache is being invalidate because project 4648e430b3354141809e7d7e8c0f10bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:36:33.740401 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-51eb1818-0221-4905-a99a-26962619a4ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:33.741166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-51eb1818-0221-4905-a99a-26962619a4ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:33.741284 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20131ba3-34bf-47ce-a77b-15897528cbe7 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:33.742857 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1257/2518] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:36:33 2022] DELETE /identity/v3/projects/4648e430b3354141809e7d7e8c0f10bb => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:36:33.747010 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69086b1-f16f-4707-b6b8-f736c6a3cc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:33.747222 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb1818-0221-4905-a99a-26962619a4ae admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:33.747654 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69086b1-f16f-4707-b6b8-f736c6a3cc4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:33.747930 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69086b1-f16f-4707-b6b8-f736c6a3cc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:33.748200 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69086b1-f16f-4707-b6b8-f736c6a3cc4f admin admin] PATH_INFO: `/v3/projects/aa4bc1ecdf7543ea9777b69428329986/users/a9c034fcd0ed4825b5661a7bc3d37719/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:33.748817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-51eb1818-0221-4905-a99a-26962619a4ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:33.761584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb1818-0221-4905-a99a-26962619a4ae admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:33.762790 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69086b1-f16f-4707-b6b8-f736c6a3cc4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa4bc1ecdf7543ea9777b69428329986, user_id=a9c034fcd0ed4825b5661a7bc3d37719, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:33.765410 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69086b1-f16f-4707-b6b8-f736c6a3cc4f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:33.766620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd7a465-56d3-4520-a11c-52178305a58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:33.767319 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7a465-56d3-4520-a11c-52178305a58e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:33.767595 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7a465-56d3-4520-a11c-52178305a58e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:33.767868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7a465-56d3-4520-a11c-52178305a58e admin admin] PATH_INFO: `/v3/users/8dd3a3efe7274c5bad82cf3b0bfd5167` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:33.768978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd7a465-56d3-4520-a11c-52178305a58e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dd3a3efe7274c5bad82cf3b0bfd5167)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:33.770902 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd7a465-56d3-4520-a11c-52178305a58e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:33.837514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1262/2519] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:36:33 2022] PUT /identity/v3/projects/aa4bc1ecdf7543ea9777b69428329986/users/a9c034fcd0ed4825b5661a7bc3d37719/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:36:33.843423 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa3ffd-438d-4aab-8270-46faa8b8a53b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:33.843720 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa3ffd-438d-4aab-8270-46faa8b8a53b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:33.844000 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa3ffd-438d-4aab-8270-46faa8b8a53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:33.863728 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f0fa3ffd-438d-4aab-8270-46faa8b8a53b None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:33.865061 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f0fa3ffd-438d-4aab-8270-46faa8b8a53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:33.924348 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1263/2520] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:36:33.933764 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa3ffd-438d-4aab-8270-46faa8b8a53b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:33.935115 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f0fa3ffd-438d-4aab-8270-46faa8b8a53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:33.961113 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ffd7a465-56d3-4520-a11c-52178305a58e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dd3a3efe7274c5bad82cf3b0bfd5167'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:33.991092 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa3ffd-438d-4aab-8270-46faa8b8a53b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:33.994621 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e9089d-0dc4-4f15-9f84-e9bc15d58c00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:33.995296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9089d-0dc4-4f15-9f84-e9bc15d58c00 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:33.995577 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9089d-0dc4-4f15-9f84-e9bc15d58c00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:33.995851 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9089d-0dc4-4f15-9f84-e9bc15d58c00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:34.004822 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e9089d-0dc4-4f15-9f84-e9bc15d58c00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:34.006230 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e9089d-0dc4-4f15-9f84-e9bc15d58c00 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:34.012967 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:34.017610 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1264/2521] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:36:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:36:34.030782 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ffd7a465-56d3-4520-a11c-52178305a58e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dd3a3efe7274c5bad82cf3b0bfd5167'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:34.031417 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ffd7a465-56d3-4520-a11c-52178305a58e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dd3a3efe7274c5bad82cf3b0bfd5167'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:34.032040 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ffd7a465-56d3-4520-a11c-52178305a58e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dd3a3efe7274c5bad82cf3b0bfd5167'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:34.044951 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ffd7a465-56d3-4520-a11c-52178305a58e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dd3a3efe7274c5bad82cf3b0bfd5167'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:34.057239 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1258/2522] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:36:33 2022] DELETE /identity/v3/users/8dd3a3efe7274c5bad82cf3b0bfd5167 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:36:34.598223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e9089d-0dc4-4f15-9f84-e9bc15d58c00 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:34.608582 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e9089d-0dc4-4f15-9f84-e9bc15d58c00 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:34.671781 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e7913-5489-4cb6-b35d-2554903457cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:34.672657 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e7913-5489-4cb6-b35d-2554903457cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:34.672957 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e7913-5489-4cb6-b35d-2554903457cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:34.673259 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e7913-5489-4cb6-b35d-2554903457cc admin admin] PATH_INFO: `/v3/projects/2c0d2c395a864f3d957e30984f9ed76e` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:34.674394 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e7913-5489-4cb6-b35d-2554903457cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c0d2c395a864f3d957e30984f9ed76e)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:34.676481 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e7913-5489-4cb6-b35d-2554903457cc admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:34.786618 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-744e7913-5489-4cb6-b35d-2554903457cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c0d2c395a864f3d957e30984f9ed76e', 'request_id': 'req-744e7913-5489-4cb6-b35d-2554903457cc'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:34.866504 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-744e7913-5489-4cb6-b35d-2554903457cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c0d2c395a864f3d957e30984f9ed76e', 'request_id': 'req-744e7913-5489-4cb6-b35d-2554903457cc'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:34.872823 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-744e7913-5489-4cb6-b35d-2554903457cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c0d2c395a864f3d957e30984f9ed76e', 'request_id': 'req-744e7913-5489-4cb6-b35d-2554903457cc'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:34.885457 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-744e7913-5489-4cb6-b35d-2554903457cc admin admin] The token cache is being invalidate because project 2c0d2c395a864f3d957e30984f9ed76e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:36:34.885748 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-744e7913-5489-4cb6-b35d-2554903457cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:34.886401 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-744e7913-5489-4cb6-b35d-2554903457cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:34.888389 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1265/2523] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:36:34 2022] DELETE /identity/v3/projects/2c0d2c395a864f3d957e30984f9ed76e => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:36:34.892332 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd7a465-56d3-4520-a11c-52178305a58e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:34.893766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ffd7a465-56d3-4520-a11c-52178305a58e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:34.907541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd7a465-56d3-4520-a11c-52178305a58e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:34.966182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e324d-314d-448f-ab48-fe464efbb6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:34.966865 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e324d-314d-448f-ab48-fe464efbb6dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:34.967168 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e324d-314d-448f-ab48-fe464efbb6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:34.967426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e324d-314d-448f-ab48-fe464efbb6dd admin admin] PATH_INFO: `/v3/users/bc7f12d0bf1a47ba89bdba2d338e894b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:34.968597 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e324d-314d-448f-ab48-fe464efbb6dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc7f12d0bf1a47ba89bdba2d338e894b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:34.970521 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762e324d-314d-448f-ab48-fe464efbb6dd admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:34.973058 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:35.206109 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-762e324d-314d-448f-ab48-fe464efbb6dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc7f12d0bf1a47ba89bdba2d338e894b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:35.404184 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-762e324d-314d-448f-ab48-fe464efbb6dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc7f12d0bf1a47ba89bdba2d338e894b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:35.404696 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-762e324d-314d-448f-ab48-fe464efbb6dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc7f12d0bf1a47ba89bdba2d338e894b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:35.406112 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-762e324d-314d-448f-ab48-fe464efbb6dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc7f12d0bf1a47ba89bdba2d338e894b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:35.450372 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-762e324d-314d-448f-ab48-fe464efbb6dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc7f12d0bf1a47ba89bdba2d338e894b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:35.450372 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1259/2524] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:36:34 2022] DELETE /identity/v3/users/bc7f12d0bf1a47ba89bdba2d338e894b => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:36:35.565191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5904947-9c7f-48e6-a574-46950f71e36b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:35.565674 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5904947-9c7f-48e6-a574-46950f71e36b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:35.565763 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5904947-9c7f-48e6-a574-46950f71e36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:35.596813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e5904947-9c7f-48e6-a574-46950f71e36b None None] MFA Rules not processed for user `a9c034fcd0ed4825b5661a7bc3d37719`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:35.598015 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e5904947-9c7f-48e6-a574-46950f71e36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:35.606761 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:35.669830 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1266/2525] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:36:35.685291 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e324d-314d-448f-ab48-fe464efbb6dd admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:35.686975 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-762e324d-314d-448f-ab48-fe464efbb6dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:35.754326 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e324d-314d-448f-ab48-fe464efbb6dd admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:35.758126 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af497c77-b250-4e22-89a0-e9f1ae408949 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:35.758768 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af497c77-b250-4e22-89a0-e9f1ae408949 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:35.759058 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af497c77-b250-4e22-89a0-e9f1ae408949 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:35.759334 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af497c77-b250-4e22-89a0-e9f1ae408949 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:35.768578 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af497c77-b250-4e22-89a0-e9f1ae408949 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:35.770012 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af497c77-b250-4e22-89a0-e9f1ae408949 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:35.779063 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1260/2526] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:36:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:36:35.882108 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5904947-9c7f-48e6-a574-46950f71e36b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:35.884594 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e5904947-9c7f-48e6-a574-46950f71e36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:35.958493 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5904947-9c7f-48e6-a574-46950f71e36b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:35.963061 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881130bf-2649-4898-b6ea-384b540313a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:35.963694 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881130bf-2649-4898-b6ea-384b540313a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:35.963979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881130bf-2649-4898-b6ea-384b540313a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:35.964258 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881130bf-2649-4898-b6ea-384b540313a7 admin admin] PATH_INFO: `/v3/projects/c7aa63907fcf4f2a8d26261154ddb930` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:35.965398 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881130bf-2649-4898-b6ea-384b540313a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7aa63907fcf4f2a8d26261154ddb930)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:35.967377 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881130bf-2649-4898-b6ea-384b540313a7 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:36.088995 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-881130bf-2649-4898-b6ea-384b540313a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7aa63907fcf4f2a8d26261154ddb930', 'request_id': 'req-881130bf-2649-4898-b6ea-384b540313a7'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:36.140765 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-881130bf-2649-4898-b6ea-384b540313a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7aa63907fcf4f2a8d26261154ddb930', 'request_id': 'req-881130bf-2649-4898-b6ea-384b540313a7'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:36.146899 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-881130bf-2649-4898-b6ea-384b540313a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7aa63907fcf4f2a8d26261154ddb930', 'request_id': 'req-881130bf-2649-4898-b6ea-384b540313a7'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:36.158378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-881130bf-2649-4898-b6ea-384b540313a7 admin admin] The token cache is being invalidate because project c7aa63907fcf4f2a8d26261154ddb930 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:36:36.158696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-881130bf-2649-4898-b6ea-384b540313a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:36.159461 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-881130bf-2649-4898-b6ea-384b540313a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:36:36.161054 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1267/2527] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:36:35 2022] DELETE /identity/v3/projects/c7aa63907fcf4f2a8d26261154ddb930 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:36:36.165995 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fe078-8bdd-421b-84fd-762802a23b26 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:36.166496 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fe078-8bdd-421b-84fd-762802a23b26 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:36.166607 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fe078-8bdd-421b-84fd-762802a23b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:36.187747 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-791fe078-8bdd-421b-84fd-762802a23b26 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:36.188967 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-791fe078-8bdd-421b-84fd-762802a23b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:36.247988 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1261/2528] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:36:36.256033 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836f8ac-62c8-40de-b52a-fcae5debb86d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:36.256322 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836f8ac-62c8-40de-b52a-fcae5debb86d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:36.256631 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836f8ac-62c8-40de-b52a-fcae5debb86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:36.277198 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5836f8ac-62c8-40de-b52a-fcae5debb86d None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:36.278412 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5836f8ac-62c8-40de-b52a-fcae5debb86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:36.286403 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1268/2529] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:36:36.290680 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791fe078-8bdd-421b-84fd-762802a23b26 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:36.301016 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791fe078-8bdd-421b-84fd-762802a23b26 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:36.305620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419a94b-867e-4d46-9ecf-9b52f126844b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:36.306222 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419a94b-867e-4d46-9ecf-9b52f126844b admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:36.306504 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419a94b-867e-4d46-9ecf-9b52f126844b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:36.306817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9419a94b-867e-4d46-9ecf-9b52f126844b admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:36.307474 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9419a94b-867e-4d46-9ecf-9b52f126844b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:36:36.307806 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9419a94b-867e-4d46-9ecf-9b52f126844b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:36.309995 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9419a94b-867e-4d46-9ecf-9b52f126844b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:36.321697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1262/2530] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:36:36.325927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5836f8ac-62c8-40de-b52a-fcae5debb86d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:36.335296 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5836f8ac-62c8-40de-b52a-fcae5debb86d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:36.340567 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2903131-bb34-4ebd-8c14-6eeb7780bb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:36.341158 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2903131-bb34-4ebd-8c14-6eeb7780bb14 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:36.341433 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2903131-bb34-4ebd-8c14-6eeb7780bb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:36.341703 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2903131-bb34-4ebd-8c14-6eeb7780bb14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:36.342385 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2903131-bb34-4ebd-8c14-6eeb7780bb14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:36.344325 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2903131-bb34-4ebd-8c14-6eeb7780bb14 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:36.473447 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1269/2531] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:36:36 2022] POST /identity/v3/projects => generated 383 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:36:36.478113 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419a94b-867e-4d46-9ecf-9b52f126844b admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:36.488429 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9419a94b-867e-4d46-9ecf-9b52f126844b admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:36.498432 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:36.551365 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf6e0f-bb07-4238-b67d-4db06dd9d3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:36.551975 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf6e0f-bb07-4238-b67d-4db06dd9d3a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:36.552254 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf6e0f-bb07-4238-b67d-4db06dd9d3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:36.552522 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf6e0f-bb07-4238-b67d-4db06dd9d3a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:36.553162 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf6e0f-bb07-4238-b67d-4db06dd9d3a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:36.555221 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf6e0f-bb07-4238-b67d-4db06dd9d3a5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:36.680862 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1263/2532] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:36:36 2022] POST /identity/v3/users => generated 292 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:36:36.681750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2903131-bb34-4ebd-8c14-6eeb7780bb14 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:36.691906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2903131-bb34-4ebd-8c14-6eeb7780bb14 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:36.696440 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f0dad-83b3-45f9-a189-c2511a96beef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:36.697030 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f0dad-83b3-45f9-a189-c2511a96beef admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:36.697310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f0dad-83b3-45f9-a189-c2511a96beef admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:36.697577 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f0dad-83b3-45f9-a189-c2511a96beef admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:36.698213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8f0dad-83b3-45f9-a189-c2511a96beef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:36.701320 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8f0dad-83b3-45f9-a189-c2511a96beef admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:36.706973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:36.712047 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1270/2533] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:36:36.716620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf6e0f-bb07-4238-b67d-4db06dd9d3a5 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:36.726525 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf6e0f-bb07-4238-b67d-4db06dd9d3a5 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:36.731309 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17af18a-b7a5-4de3-83d4-7a605a20016e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:36.731901 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17af18a-b7a5-4de3-83d4-7a605a20016e admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:36.732185 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17af18a-b7a5-4de3-83d4-7a605a20016e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:36.732458 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17af18a-b7a5-4de3-83d4-7a605a20016e admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:36.733100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17af18a-b7a5-4de3-83d4-7a605a20016e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:36.734692 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17af18a-b7a5-4de3-83d4-7a605a20016e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:36.745115 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1264/2534] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:36:36.749160 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f0dad-83b3-45f9-a189-c2511a96beef admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:36.759614 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f0dad-83b3-45f9-a189-c2511a96beef admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:36.765279 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac965ee2-4b6d-49a1-865c-ba3940b3ce66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:36.765917 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac965ee2-4b6d-49a1-865c-ba3940b3ce66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:36.766195 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac965ee2-4b6d-49a1-865c-ba3940b3ce66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:36.766475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac965ee2-4b6d-49a1-865c-ba3940b3ce66 admin admin] PATH_INFO: `/v3/projects/7232d6277f26496ab552afa4abe13ad5/users/889d4c60870447bda2926106026a0537/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:36.781328 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac965ee2-4b6d-49a1-865c-ba3940b3ce66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7232d6277f26496ab552afa4abe13ad5, user_id=889d4c60870447bda2926106026a0537, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:36.786201 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac965ee2-4b6d-49a1-865c-ba3940b3ce66 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:36.850533 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1271/2535] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:36:36 2022] PUT /identity/v3/projects/7232d6277f26496ab552afa4abe13ad5/users/889d4c60870447bda2926106026a0537/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:36:36.855719 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc495c-02ea-4e49-8dc2-4cd85eaffc35 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:36.856137 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc495c-02ea-4e49-8dc2-4cd85eaffc35 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:36.856407 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc495c-02ea-4e49-8dc2-4cd85eaffc35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:36.876039 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-85bc495c-02ea-4e49-8dc2-4cd85eaffc35 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:36.877274 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-85bc495c-02ea-4e49-8dc2-4cd85eaffc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:36.936460 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1265/2536] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:36:36.946138 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac965ee2-4b6d-49a1-865c-ba3940b3ce66 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:36.948440 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ac965ee2-4b6d-49a1-865c-ba3940b3ce66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:37.004220 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac965ee2-4b6d-49a1-865c-ba3940b3ce66 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:37.007865 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7eb85c-d645-44c3-8bdc-81013c7fb28a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:37.011640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7eb85c-d645-44c3-8bdc-81013c7fb28a service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:37.011640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7eb85c-d645-44c3-8bdc-81013c7fb28a service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:37.011640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7eb85c-d645-44c3-8bdc-81013c7fb28a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:37.018291 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7eb85c-d645-44c3-8bdc-81013c7fb28a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:37.019967 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7eb85c-d645-44c3-8bdc-81013c7fb28a service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:37.030281 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1272/2537] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:36:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:36:38.455255 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43d5f5-5666-49f9-8ead-53597e9387b8 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:38.455812 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43d5f5-5666-49f9-8ead-53597e9387b8 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:38.455812 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43d5f5-5666-49f9-8ead-53597e9387b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:38.492716 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-da43d5f5-5666-49f9-8ead-53597e9387b8 None None] MFA Rules not processed for user `889d4c60870447bda2926106026a0537`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:38.493962 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-da43d5f5-5666-49f9-8ead-53597e9387b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:38.550922 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d2c0e-36d5-42a3-8238-7ceee0b5a370 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:38.551818 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d2c0e-36d5-42a3-8238-7ceee0b5a370 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:38.552367 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d2c0e-36d5-42a3-8238-7ceee0b5a370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:38.570133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1266/2538] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:36:38.578209 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-859d2c0e-36d5-42a3-8238-7ceee0b5a370 None None] MFA Rules not processed for user `a9c034fcd0ed4825b5661a7bc3d37719`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:38.579451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-859d2c0e-36d5-42a3-8238-7ceee0b5a370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:38.580993 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43d5f5-5666-49f9-8ead-53597e9387b8 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:36:38.582464 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-da43d5f5-5666-49f9-8ead-53597e9387b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:38.604808 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:38.636656 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1273/2539] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:36:38.640985 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43d5f5-5666-49f9-8ead-53597e9387b8 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:36:38.644631 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d995c-70cf-4982-9093-4139af786426 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:36:38.645239 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d995c-70cf-4982-9093-4139af786426 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:38.645520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d995c-70cf-4982-9093-4139af786426 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:38.645795 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d995c-70cf-4982-9093-4139af786426 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:38.655159 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d995c-70cf-4982-9093-4139af786426 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:36:38.656578 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d995c-70cf-4982-9093-4139af786426 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:36:38.665549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1267/2540] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:36:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:36:40.372250 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b94937-2c4e-4513-bc6c-d0acf648ac9f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:36:40.372810 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b94937-2c4e-4513-bc6c-d0acf648ac9f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:36:40.372810 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b94937-2c4e-4513-bc6c-d0acf648ac9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:36:40.393135 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-42b94937-2c4e-4513-bc6c-d0acf648ac9f None None] MFA Rules not processed for user `889d4c60870447bda2926106026a0537`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:36:40.394671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-42b94937-2c4e-4513-bc6c-d0acf648ac9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:36:40.402166 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1274/2541] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:36:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:36:48.672892 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=17 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:36:50.386600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=64,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:37:04.208961 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d455b2-c68b-444d-a435-f661bfa79a3a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:37:04.209552 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d455b2-c68b-444d-a435-f661bfa79a3a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:37:04.209552 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d455b2-c68b-444d-a435-f661bfa79a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:37:04.229266 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b9d455b2-c68b-444d-a435-f661bfa79a3a None None] MFA Rules not processed for user `a9c034fcd0ed4825b5661a7bc3d37719`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:37:04.230597 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b9d455b2-c68b-444d-a435-f661bfa79a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:37:04.238467 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1268/2542] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:37:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:37:14.223743 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:37:29.105598 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b94937-2c4e-4513-bc6c-d0acf648ac9f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:37:29.115130 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b94937-2c4e-4513-bc6c-d0acf648ac9f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:37:29.118615 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada6814-cdea-4baf-9a50-b2c9f0936c3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:37:29.119268 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada6814-cdea-4baf-9a50-b2c9f0936c3f service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:37:29.119550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada6814-cdea-4baf-9a50-b2c9f0936c3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:37:29.119822 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada6814-cdea-4baf-9a50-b2c9f0936c3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:37:29.121569 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-eada6814-cdea-4baf-9a50-b2c9f0936c3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:37:29.193074 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada6814-cdea-4baf-9a50-b2c9f0936c3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:37:29.194543 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada6814-cdea-4baf-9a50-b2c9f0936c3f service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:37:29.205994 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1275/2543] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:37:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:37:30.380819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d455b2-c68b-444d-a435-f661bfa79a3a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:37:30.382219 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b9d455b2-c68b-444d-a435-f661bfa79a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:37:30.394586 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d455b2-c68b-444d-a435-f661bfa79a3a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:37:30.398101 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98b441-65b6-46bb-84a8-1b688f694b39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:37:30.398697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98b441-65b6-46bb-84a8-1b688f694b39 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:37:30.399001 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98b441-65b6-46bb-84a8-1b688f694b39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:37:30.399272 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98b441-65b6-46bb-84a8-1b688f694b39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:37:30.401462 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0f98b441-65b6-46bb-84a8-1b688f694b39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:37:30.414157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98b441-65b6-46bb-84a8-1b688f694b39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:37:30.415582 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98b441-65b6-46bb-84a8-1b688f694b39 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:37:30.426982 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1269/2544] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:37:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:37:33.117497 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49525e46-7415-44f1-92fe-7dc4cb482804 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:37:33.117951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49525e46-7415-44f1-92fe-7dc4cb482804 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:37:33.118061 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49525e46-7415-44f1-92fe-7dc4cb482804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:37:33.138350 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-49525e46-7415-44f1-92fe-7dc4cb482804 None None] MFA Rules not processed for user `5233689a53364258b86ad2fa012d8db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:37:33.139590 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-49525e46-7415-44f1-92fe-7dc4cb482804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:37:33.194849 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1276/2545] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Sep 2 09:37:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:37:33.341909 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98b441-65b6-46bb-84a8-1b688f694b39 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:37:33.343334 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0f98b441-65b6-46bb-84a8-1b688f694b39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:37:33.410858 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98b441-65b6-46bb-84a8-1b688f694b39 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:37:33.415713 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4e135-8678-4bc6-8c82-edab6db42c2c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30fe233a7d4c4fbea6ede233bd9ff8c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:37:33.416987 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4e135-8678-4bc6-8c82-edab6db42c2c service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:37:33.417266 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4e135-8678-4bc6-8c82-edab6db42c2c service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:37:33.417538 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4e135-8678-4bc6-8c82-edab6db42c2c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:37:33.425590 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4e135-8678-4bc6-8c82-edab6db42c2c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:37:33.427228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4e135-8678-4bc6-8c82-edab6db42c2c service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:37:33.435162 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1270/2546] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Sep 2 09:37:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 09:37:40.022308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49525e46-7415-44f1-92fe-7dc4cb482804 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:37:40.033372 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49525e46-7415-44f1-92fe-7dc4cb482804 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:37:40.038529 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b97dc07-aa8e-47ff-8981-404493ea2824 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:37:40.039373 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97dc07-aa8e-47ff-8981-404493ea2824 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:37:40.039726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97dc07-aa8e-47ff-8981-404493ea2824 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:37:40.040109 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97dc07-aa8e-47ff-8981-404493ea2824 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:37:40.042510 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4b97dc07-aa8e-47ff-8981-404493ea2824 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:37:40.149270 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b97dc07-aa8e-47ff-8981-404493ea2824 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:37:40.151220 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b97dc07-aa8e-47ff-8981-404493ea2824 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:37:40.165092 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1277/2547] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:37:40 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:37:43.432036 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=38 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:37:50.156478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=87 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:38:40.428091 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4e135-8678-4bc6-8c82-edab6db42c2c service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:38:40.430561 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-59f4e135-8678-4bc6-8c82-edab6db42c2c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:38:40.445115 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4e135-8678-4bc6-8c82-edab6db42c2c service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:38:40.450532 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70420b34-8f15-4e92-8de0-0991c2a600d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:38:40.451161 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70420b34-8f15-4e92-8de0-0991c2a600d7 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:38:40.451444 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70420b34-8f15-4e92-8de0-0991c2a600d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:38:40.451718 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70420b34-8f15-4e92-8de0-0991c2a600d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:38:40.453347 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-70420b34-8f15-4e92-8de0-0991c2a600d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:38:40.514811 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70420b34-8f15-4e92-8de0-0991c2a600d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:38:40.516403 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70420b34-8f15-4e92-8de0-0991c2a600d7 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:38:40.527439 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1271/2548] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:38:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:38:40.949913 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b97dc07-aa8e-47ff-8981-404493ea2824 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:38:40.956168 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b97dc07-aa8e-47ff-8981-404493ea2824 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:38:40.960666 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ecceb-5de3-424a-a992-c8421d5c8d2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:38:40.962321 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ecceb-5de3-424a-a992-c8421d5c8d2d service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:38:40.962587 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ecceb-5de3-424a-a992-c8421d5c8d2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:38:40.962910 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ecceb-5de3-424a-a992-c8421d5c8d2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:38:40.964776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1b4ecceb-5de3-424a-a992-c8421d5c8d2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:38:40.974816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ecceb-5de3-424a-a992-c8421d5c8d2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:38:40.976224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ecceb-5de3-424a-a992-c8421d5c8d2d service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:38:40.984087 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1278/2549] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:38:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:38:50.522415 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=28 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:38:50.980643 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:39:04.331171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62401f-32c0-4ebf-a511-e5b32e35bdc2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:39:04.331664 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62401f-32c0-4ebf-a511-e5b32e35bdc2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:39:04.331764 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d62401f-32c0-4ebf-a511-e5b32e35bdc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:39:04.350935 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-3d62401f-32c0-4ebf-a511-e5b32e35bdc2 None None] MFA Rules not processed for user `c79aaf1f72634eefb5e89c8005afe54b`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:39:04.352125 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3d62401f-32c0-4ebf-a511-e5b32e35bdc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:39:04.360412 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1272/2550] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:39:04 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:39:10.704835 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ecceb-5de3-424a-a992-c8421d5c8d2d service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:39:10.706285 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1b4ecceb-5de3-424a-a992-c8421d5c8d2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:39:10.722958 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ecceb-5de3-424a-a992-c8421d5c8d2d service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:39:10.731907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6914bf4-84a7-47fb-8247-07806179e609 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:39:10.732534 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6914bf4-84a7-47fb-8247-07806179e609 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:39:10.732813 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6914bf4-84a7-47fb-8247-07806179e609 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:39:10.733082 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6914bf4-84a7-47fb-8247-07806179e609 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:39:10.734834 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a6914bf4-84a7-47fb-8247-07806179e609 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:39:10.747778 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6914bf4-84a7-47fb-8247-07806179e609 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:39:10.749274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6914bf4-84a7-47fb-8247-07806179e609 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:39:10.759922 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1279/2551] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:39:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:39:14.345425 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:39:14.692812 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62401f-32c0-4ebf-a511-e5b32e35bdc2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:39:14.706871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d62401f-32c0-4ebf-a511-e5b32e35bdc2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:39:14.713096 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed853f-63f4-4100-af4d-8928107ea8ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:39:14.714483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed853f-63f4-4100-af4d-8928107ea8ec service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:39:14.714782 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed853f-63f4-4100-af4d-8928107ea8ec service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:39:14.715238 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed853f-63f4-4100-af4d-8928107ea8ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:39:14.717082 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-79ed853f-63f4-4100-af4d-8928107ea8ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:39:14.730130 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed853f-63f4-4100-af4d-8928107ea8ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:39:14.731897 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ed853f-63f4-4100-af4d-8928107ea8ec service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:39:14.755145 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1273/2552] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:39:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:39:20.755839 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:39:24.739237 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:39:49.185667 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6914bf4-84a7-47fb-8247-07806179e609 service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:39:49.196725 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6914bf4-84a7-47fb-8247-07806179e609 service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:39:49.202668 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0168d5a-a92b-4836-987e-3bad88fe6b8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:39:49.203280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0168d5a-a92b-4836-987e-3bad88fe6b8f service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:39:49.203556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0168d5a-a92b-4836-987e-3bad88fe6b8f service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:39:49.203823 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0168d5a-a92b-4836-987e-3bad88fe6b8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:39:49.205490 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a0168d5a-a92b-4836-987e-3bad88fe6b8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:39:49.266324 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0168d5a-a92b-4836-987e-3bad88fe6b8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:39:49.267961 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0168d5a-a92b-4836-987e-3bad88fe6b8f service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:39:49.277069 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1280/2553] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:39:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:39:59.272918 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=28 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:41:43.275304 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed853f-63f4-4100-af4d-8928107ea8ec service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:41:43.276977 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-79ed853f-63f4-4100-af4d-8928107ea8ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:41:43.411942 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed853f-63f4-4100-af4d-8928107ea8ec service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:41:43.417020 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd6a25-ce50-44f0-b6f3-74d4acc6203e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5177fa4d168c4c219e09c7178bbbc790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:41:43.417626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd6a25-ce50-44f0-b6f3-74d4acc6203e service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:41:43.417907 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd6a25-ce50-44f0-b6f3-74d4acc6203e service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:41:43.418183 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd6a25-ce50-44f0-b6f3-74d4acc6203e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:41:43.419908 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7ecd6a25-ce50-44f0-b6f3-74d4acc6203e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:41:43.431031 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd6a25-ce50-44f0-b6f3-74d4acc6203e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:41:43.432597 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd6a25-ce50-44f0-b6f3-74d4acc6203e service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:41:43.442666 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1274/2554] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:41:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:41:53.438577 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=63 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:42:43.568091 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0168d5a-a92b-4836-987e-3bad88fe6b8f service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:42:43.599211 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0168d5a-a92b-4836-987e-3bad88fe6b8f service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:42:43.604586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297b674-a2e1-4880-aa61-8ffa548299d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:42:43.605189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297b674-a2e1-4880-aa61-8ffa548299d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:42:43.605547 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297b674-a2e1-4880-aa61-8ffa548299d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:42:43.605827 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f297b674-a2e1-4880-aa61-8ffa548299d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:42:43.623842 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297b674-a2e1-4880-aa61-8ffa548299d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:42:43.625257 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f297b674-a2e1-4880-aa61-8ffa548299d9 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:42:43.646747 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1281/2555] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:42:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:42:43.653886 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd6a25-ce50-44f0-b6f3-74d4acc6203e service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:42:43.662951 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd6a25-ce50-44f0-b6f3-74d4acc6203e service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:42:43.666451 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81547e2b-1e18-4c52-8a12-aa193a99cd42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:42:43.667074 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81547e2b-1e18-4c52-8a12-aa193a99cd42 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:42:43.667354 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81547e2b-1e18-4c52-8a12-aa193a99cd42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:42:43.667627 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81547e2b-1e18-4c52-8a12-aa193a99cd42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:42:43.678352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81547e2b-1e18-4c52-8a12-aa193a99cd42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:42:43.679865 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81547e2b-1e18-4c52-8a12-aa193a99cd42 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:42:43.706653 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1275/2556] 10.222.0.57 () {60 vars in 1396 bytes} [Fri Sep 2 09:42:43 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3270 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:42:43.831850 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297b674-a2e1-4880-aa61-8ffa548299d9 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:42:43.840323 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f297b674-a2e1-4880-aa61-8ffa548299d9 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:42:43.844977 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5984f-cfc1-45fe-be22-bd6ff8b35a19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:42:43.846369 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5984f-cfc1-45fe-be22-bd6ff8b35a19 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:42:43.846369 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5984f-cfc1-45fe-be22-bd6ff8b35a19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:42:43.846686 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5984f-cfc1-45fe-be22-bd6ff8b35a19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:42:43.848603 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e2d5984f-cfc1-45fe-be22-bd6ff8b35a19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:42:43.859954 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5984f-cfc1-45fe-be22-bd6ff8b35a19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:42:43.862039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5984f-cfc1-45fe-be22-bd6ff8b35a19 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:42:43.870855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1282/2557] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:42:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:42:43.957637 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81547e2b-1e18-4c52-8a12-aa193a99cd42 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:42:43.966665 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81547e2b-1e18-4c52-8a12-aa193a99cd42 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:42:43.970402 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7df8d-2ec6-4c8a-993e-b45352fd924d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:42:43.971619 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7df8d-2ec6-4c8a-993e-b45352fd924d service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:42:43.971899 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7df8d-2ec6-4c8a-993e-b45352fd924d service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:42:43.972169 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7df8d-2ec6-4c8a-993e-b45352fd924d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:42:43.981799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7df8d-2ec6-4c8a-993e-b45352fd924d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:42:43.983263 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7df8d-2ec6-4c8a-993e-b45352fd924d service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:42:43.993418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1276/2558] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:42:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:42:44.022528 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5984f-cfc1-45fe-be22-bd6ff8b35a19 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:42:44.033639 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5984f-cfc1-45fe-be22-bd6ff8b35a19 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:42:44.038940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d62e4-64b1-435e-952d-c4db004931c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:42:44.039543 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d62e4-64b1-435e-952d-c4db004931c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:42:44.039822 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d62e4-64b1-435e-952d-c4db004931c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:42:44.040095 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d62e4-64b1-435e-952d-c4db004931c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:42:44.052355 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d62e4-64b1-435e-952d-c4db004931c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:42:44.053825 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d62e4-64b1-435e-952d-c4db004931c7 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:42:44.067150 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1283/2559] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:42:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:42:53.989527 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=19 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:42:54.061341 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=34 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:43:45.083359 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7df8d-2ec6-4c8a-993e-b45352fd924d service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:43:45.101959 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7df8d-2ec6-4c8a-993e-b45352fd924d service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:43:45.105940 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93046ee9-e1ec-44e6-8576-dcdb74631202 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:43:45.106525 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93046ee9-e1ec-44e6-8576-dcdb74631202 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:43:45.106832 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93046ee9-e1ec-44e6-8576-dcdb74631202 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:43:45.107101 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93046ee9-e1ec-44e6-8576-dcdb74631202 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:43:45.114176 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93046ee9-e1ec-44e6-8576-dcdb74631202 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:43:45.115667 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93046ee9-e1ec-44e6-8576-dcdb74631202 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:43:45.124235 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1277/2560] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:43:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:43:55.121093 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=10 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:44:44.363565 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d62e4-64b1-435e-952d-c4db004931c7 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:44:44.372188 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d62e4-64b1-435e-952d-c4db004931c7 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:44:44.375900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1d85c-1ab2-4e85-b054-c9eaa16c1b18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:44:44.376494 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1d85c-1ab2-4e85-b054-c9eaa16c1b18 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:44:44.376771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1d85c-1ab2-4e85-b054-c9eaa16c1b18 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:44:44.377039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1d85c-1ab2-4e85-b054-c9eaa16c1b18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:44:44.386313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a1d85c-1ab2-4e85-b054-c9eaa16c1b18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:44:44.387760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a1d85c-1ab2-4e85-b054-c9eaa16c1b18 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:44:44.420556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1284/2561] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:44:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:44:44.604784 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc27c9-f6c5-469f-b115-6db4337537fa None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:44:44.605285 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc27c9-f6c5-469f-b115-6db4337537fa None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:44:44.605477 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc27c9-f6c5-469f-b115-6db4337537fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:44:44.668924 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ebcc27c9-f6c5-469f-b115-6db4337537fa None None] MFA Rules not processed for user `5233689a53364258b86ad2fa012d8db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:44:44.670606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ebcc27c9-f6c5-469f-b115-6db4337537fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:44:44.690095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1278/2562] 10.222.0.57 () {58 vars in 967 bytes} [Fri Sep 2 09:44:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:44:44.821373 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1d85c-1ab2-4e85-b054-c9eaa16c1b18 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:44:44.843106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1d85c-1ab2-4e85-b054-c9eaa16c1b18 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:44:44.848928 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040c121-0989-4c78-b6c7-0e909d9db3e7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30fe233a7d4c4fbea6ede233bd9ff8c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:44:44.849553 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040c121-0989-4c78-b6c7-0e909d9db3e7 service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:44:44.849817 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040c121-0989-4c78-b6c7-0e909d9db3e7 service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:44:44.850092 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040c121-0989-4c78-b6c7-0e909d9db3e7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:44:44.858597 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040c121-0989-4c78-b6c7-0e909d9db3e7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:44:44.860211 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040c121-0989-4c78-b6c7-0e909d9db3e7 service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:44:44.868796 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1285/2563] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Sep 2 09:44:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 09:44:54.687190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=22 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:44:54.865401 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=23 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:45:15.423150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc27c9-f6c5-469f-b115-6db4337537fa None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:45:15.435503 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc27c9-f6c5-469f-b115-6db4337537fa None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:45:15.441074 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d8720-da4b-491c-9a7e-20bc64845b06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:45:15.441675 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d8720-da4b-491c-9a7e-20bc64845b06 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:45:15.441953 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d8720-da4b-491c-9a7e-20bc64845b06 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:45:15.442230 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96d8720-da4b-491c-9a7e-20bc64845b06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:45:15.450377 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d8720-da4b-491c-9a7e-20bc64845b06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:45:15.452176 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96d8720-da4b-491c-9a7e-20bc64845b06 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:45:15.482221 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1279/2564] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:45:15 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:45:15.501875 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040c121-0989-4c78-b6c7-0e909d9db3e7 service swift] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:45:15.511444 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040c121-0989-4c78-b6c7-0e909d9db3e7 service swift] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:45:15.515165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b447f49-aa12-4007-8b3c-afa582923e38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:45:15.515771 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b447f49-aa12-4007-8b3c-afa582923e38 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:45:15.516048 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b447f49-aa12-4007-8b3c-afa582923e38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:45:15.516315 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b447f49-aa12-4007-8b3c-afa582923e38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:45:15.518029 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9b447f49-aa12-4007-8b3c-afa582923e38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:45:15.555821 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b447f49-aa12-4007-8b3c-afa582923e38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:45:15.559347 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b447f49-aa12-4007-8b3c-afa582923e38 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:45:15.576748 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1286/2565] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:45:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:45:16.153997 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d8720-da4b-491c-9a7e-20bc64845b06 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:45:16.165800 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96d8720-da4b-491c-9a7e-20bc64845b06 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:45:16.170948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eebe1c-a22d-4226-b6e2-07cb1173df92 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5177fa4d168c4c219e09c7178bbbc790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:45:16.171785 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eebe1c-a22d-4226-b6e2-07cb1173df92 service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:45:16.172064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eebe1c-a22d-4226-b6e2-07cb1173df92 service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:45:16.172358 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eebe1c-a22d-4226-b6e2-07cb1173df92 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:45:16.174125 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e9eebe1c-a22d-4226-b6e2-07cb1173df92 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:45:16.186397 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eebe1c-a22d-4226-b6e2-07cb1173df92 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:45:16.187871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eebe1c-a22d-4226-b6e2-07cb1173df92 service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:45:16.198715 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1280/2566] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:45:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:45:16.304135 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b447f49-aa12-4007-8b3c-afa582923e38 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:45:16.305574 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9b447f49-aa12-4007-8b3c-afa582923e38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:45:16.322372 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b447f49-aa12-4007-8b3c-afa582923e38 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:45:16.327056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded01be2-0b6d-4090-8a81-ac0f0f0b3501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:45:16.327681 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded01be2-0b6d-4090-8a81-ac0f0f0b3501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:45:16.327959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded01be2-0b6d-4090-8a81-ac0f0f0b3501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:45:16.328233 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded01be2-0b6d-4090-8a81-ac0f0f0b3501 admin admin] PATH_INFO: `/v3/users/a9c034fcd0ed4825b5661a7bc3d37719` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:45:16.329419 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded01be2-0b6d-4090-8a81-ac0f0f0b3501 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9c034fcd0ed4825b5661a7bc3d37719)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:45:16.331460 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded01be2-0b6d-4090-8a81-ac0f0f0b3501 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:45:16.499823 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ded01be2-0b6d-4090-8a81-ac0f0f0b3501 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9c034fcd0ed4825b5661a7bc3d37719'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:45:16.506651 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ded01be2-0b6d-4090-8a81-ac0f0f0b3501 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9c034fcd0ed4825b5661a7bc3d37719'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:45:16.559938 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ded01be2-0b6d-4090-8a81-ac0f0f0b3501 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9c034fcd0ed4825b5661a7bc3d37719'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:45:16.560554 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ded01be2-0b6d-4090-8a81-ac0f0f0b3501 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9c034fcd0ed4825b5661a7bc3d37719'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:45:16.572699 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ded01be2-0b6d-4090-8a81-ac0f0f0b3501 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9c034fcd0ed4825b5661a7bc3d37719'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:45:16.578623 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1287/2567] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:45:16 2022] DELETE /identity/v3/users/a9c034fcd0ed4825b5661a7bc3d37719 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:45:16.812064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eebe1c-a22d-4226-b6e2-07cb1173df92 service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:45:16.824094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eebe1c-a22d-4226-b6e2-07cb1173df92 service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:45:16.882861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176d533f-56fb-449b-b88e-ddcc258a5e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:45:16.883497 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d533f-56fb-449b-b88e-ddcc258a5e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:45:16.883782 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d533f-56fb-449b-b88e-ddcc258a5e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:45:16.884129 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176d533f-56fb-449b-b88e-ddcc258a5e06 admin admin] PATH_INFO: `/v3/projects/aa4bc1ecdf7543ea9777b69428329986` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:45:16.885175 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d533f-56fb-449b-b88e-ddcc258a5e06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa4bc1ecdf7543ea9777b69428329986)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:45:16.887105 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176d533f-56fb-449b-b88e-ddcc258a5e06 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:45:16.983775 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-176d533f-56fb-449b-b88e-ddcc258a5e06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa4bc1ecdf7543ea9777b69428329986', 'request_id': 'req-176d533f-56fb-449b-b88e-ddcc258a5e06'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:45:16.989361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-176d533f-56fb-449b-b88e-ddcc258a5e06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa4bc1ecdf7543ea9777b69428329986', 'request_id': 'req-176d533f-56fb-449b-b88e-ddcc258a5e06'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:45:16.995478 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-176d533f-56fb-449b-b88e-ddcc258a5e06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa4bc1ecdf7543ea9777b69428329986', 'request_id': 'req-176d533f-56fb-449b-b88e-ddcc258a5e06'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:45:17.079895 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-176d533f-56fb-449b-b88e-ddcc258a5e06 admin admin] The token cache is being invalidate because project aa4bc1ecdf7543ea9777b69428329986 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:45:17.080122 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-176d533f-56fb-449b-b88e-ddcc258a5e06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:45:17.080763 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-176d533f-56fb-449b-b88e-ddcc258a5e06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:45:17.082942 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1281/2568] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:45:16 2022] DELETE /identity/v3/projects/aa4bc1ecdf7543ea9777b69428329986 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:45:17.088718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2960b9e-22ef-46f2-a049-7961eee0c9af None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:45:17.089005 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2960b9e-22ef-46f2-a049-7961eee0c9af None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:45:17.089281 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2960b9e-22ef-46f2-a049-7961eee0c9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:45:17.151017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e2960b9e-22ef-46f2-a049-7961eee0c9af None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:45:17.152491 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e2960b9e-22ef-46f2-a049-7961eee0c9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:45:17.202594 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:45:17.211324 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1288/2569] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:45:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 09:45:17.220570 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d65b31-9cc4-4054-8f9b-e5906c0ae47c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:45:17.220856 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d65b31-9cc4-4054-8f9b-e5906c0ae47c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:45:17.221129 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d65b31-9cc4-4054-8f9b-e5906c0ae47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:45:17.240843 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-44d65b31-9cc4-4054-8f9b-e5906c0ae47c None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:45:17.248339 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-44d65b31-9cc4-4054-8f9b-e5906c0ae47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:45:17.249970 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1282/2570] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:45:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:45:17.254679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2960b9e-22ef-46f2-a049-7961eee0c9af None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:45:17.264730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2960b9e-22ef-46f2-a049-7961eee0c9af None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:45:17.270040 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aed475-8229-4d37-a4e0-aa09a0c8eb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:45:17.270637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aed475-8229-4d37-a4e0-aa09a0c8eb33 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:45:17.270953 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aed475-8229-4d37-a4e0-aa09a0c8eb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:45:17.271252 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aed475-8229-4d37-a4e0-aa09a0c8eb33 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:45:17.271972 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aed475-8229-4d37-a4e0-aa09a0c8eb33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:45:17.272270 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aed475-8229-4d37-a4e0-aa09a0c8eb33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:45:17.273912 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aed475-8229-4d37-a4e0-aa09a0c8eb33 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:45:17.285753 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1289/2571] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:45:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 09:45:17.289924 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d65b31-9cc4-4054-8f9b-e5906c0ae47c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:45:17.299115 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d65b31-9cc4-4054-8f9b-e5906c0ae47c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:45:17.303800 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e70e65-334e-435f-930f-7b7652d9fc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:45:17.318973 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e70e65-334e-435f-930f-7b7652d9fc4c admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:45:17.318973 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e70e65-334e-435f-930f-7b7652d9fc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:45:17.318973 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e70e65-334e-435f-930f-7b7652d9fc4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:45:17.318973 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e70e65-334e-435f-930f-7b7652d9fc4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:45:17.318973 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e70e65-334e-435f-930f-7b7652d9fc4c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:45:17.401308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1283/2572] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:45:17 2022] POST /identity/v3/projects => generated 389 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:45:17.405811 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aed475-8229-4d37-a4e0-aa09a0c8eb33 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:45:17.416640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aed475-8229-4d37-a4e0-aa09a0c8eb33 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:45:17.473727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b26dc44-13b3-4d37-b19e-4e7db55d5f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:45:17.474321 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26dc44-13b3-4d37-b19e-4e7db55d5f43 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:45:17.474601 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26dc44-13b3-4d37-b19e-4e7db55d5f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:45:17.474906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b26dc44-13b3-4d37-b19e-4e7db55d5f43 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:45:17.475533 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b26dc44-13b3-4d37-b19e-4e7db55d5f43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:45:17.477781 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b26dc44-13b3-4d37-b19e-4e7db55d5f43 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:45:17.553055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1290/2573] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:45:17 2022] POST /identity/v3/users => generated 295 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:45:17.557228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e70e65-334e-435f-930f-7b7652d9fc4c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:45:17.567826 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e70e65-334e-435f-930f-7b7652d9fc4c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:45:17.572534 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d767888b-cdea-4c6d-af0d-59cc49920d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:45:17.573140 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767888b-cdea-4c6d-af0d-59cc49920d63 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:45:17.573423 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767888b-cdea-4c6d-af0d-59cc49920d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:45:17.573699 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767888b-cdea-4c6d-af0d-59cc49920d63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:45:17.574344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d767888b-cdea-4c6d-af0d-59cc49920d63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:45:17.576184 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d767888b-cdea-4c6d-af0d-59cc49920d63 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:45:17.582024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:45:17.587054 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1284/2574] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:45:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:45:17.590822 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b26dc44-13b3-4d37-b19e-4e7db55d5f43 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:45:17.601356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b26dc44-13b3-4d37-b19e-4e7db55d5f43 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:45:17.605981 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec9cf3c-82f5-4735-a9de-50c6265c3a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:45:17.606608 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9cf3c-82f5-4735-a9de-50c6265c3a63 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:45:17.606926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9cf3c-82f5-4735-a9de-50c6265c3a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:45:17.607206 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec9cf3c-82f5-4735-a9de-50c6265c3a63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:45:17.607856 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec9cf3c-82f5-4735-a9de-50c6265c3a63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:45:17.609512 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec9cf3c-82f5-4735-a9de-50c6265c3a63 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:45:17.620069 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1291/2575] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:45:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:45:17.624040 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d767888b-cdea-4c6d-af0d-59cc49920d63 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:45:17.633963 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d767888b-cdea-4c6d-af0d-59cc49920d63 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:45:17.638684 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8989b5-e407-4523-b94b-3d4103d3c7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:45:17.639378 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8989b5-e407-4523-b94b-3d4103d3c7e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:45:17.639663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8989b5-e407-4523-b94b-3d4103d3c7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:45:17.639942 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8989b5-e407-4523-b94b-3d4103d3c7e6 admin admin] PATH_INFO: `/v3/projects/3b21d4d29aae43889d1904dbca72c88f/users/cc916c79ed69429e9b6d4f2e5405be41/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:45:17.656081 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8989b5-e407-4523-b94b-3d4103d3c7e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b21d4d29aae43889d1904dbca72c88f, user_id=cc916c79ed69429e9b6d4f2e5405be41, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:45:17.658645 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8989b5-e407-4523-b94b-3d4103d3c7e6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:45:17.720019 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1285/2576] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:45:17 2022] PUT /identity/v3/projects/3b21d4d29aae43889d1904dbca72c88f/users/cc916c79ed69429e9b6d4f2e5405be41/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:45:17.734521 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22f5da-6e8c-421f-85e3-7822cf720696 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:45:17.734840 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22f5da-6e8c-421f-85e3-7822cf720696 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:45:17.735121 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22f5da-6e8c-421f-85e3-7822cf720696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:45:17.763052 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5d22f5da-6e8c-421f-85e3-7822cf720696 None None] MFA Rules not processed for user `cc916c79ed69429e9b6d4f2e5405be41`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:45:17.764294 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5d22f5da-6e8c-421f-85e3-7822cf720696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:45:17.826812 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1292/2577] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:45:17 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:45:27.721570 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:45:27.823957 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=72,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:46:17.893187 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8989b5-e407-4523-b94b-3d4103d3c7e6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:46:17.954528 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8989b5-e407-4523-b94b-3d4103d3c7e6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:46:17.959122 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c269e04-3679-4f6b-9ddb-2aef3666bde9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:46:17.959745 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c269e04-3679-4f6b-9ddb-2aef3666bde9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:46:17.960076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c269e04-3679-4f6b-9ddb-2aef3666bde9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:46:17.960863 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c269e04-3679-4f6b-9ddb-2aef3666bde9 admin admin] PATH_INFO: `/v3/users/cc916c79ed69429e9b6d4f2e5405be41` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:46:17.961989 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c269e04-3679-4f6b-9ddb-2aef3666bde9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc916c79ed69429e9b6d4f2e5405be41)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:46:17.963954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c269e04-3679-4f6b-9ddb-2aef3666bde9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:46:18.099644 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1c269e04-3679-4f6b-9ddb-2aef3666bde9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc916c79ed69429e9b6d4f2e5405be41'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:46:18.142087 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1c269e04-3679-4f6b-9ddb-2aef3666bde9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc916c79ed69429e9b6d4f2e5405be41'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:46:18.142766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1c269e04-3679-4f6b-9ddb-2aef3666bde9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc916c79ed69429e9b6d4f2e5405be41'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:46:18.143417 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1c269e04-3679-4f6b-9ddb-2aef3666bde9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc916c79ed69429e9b6d4f2e5405be41'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:46:18.155844 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-1c269e04-3679-4f6b-9ddb-2aef3666bde9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc916c79ed69429e9b6d4f2e5405be41'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:46:18.167788 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1286/2578] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:46:17 2022] DELETE /identity/v3/users/cc916c79ed69429e9b6d4f2e5405be41 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:46:18.172207 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92853da4-1657-4633-9aac-47d80571417f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:46:18.172714 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92853da4-1657-4633-9aac-47d80571417f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:46:18.172832 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92853da4-1657-4633-9aac-47d80571417f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:46:18.191633 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-92853da4-1657-4633-9aac-47d80571417f None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:46:18.192835 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-92853da4-1657-4633-9aac-47d80571417f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:46:18.249688 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1293/2579] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:46:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:46:18.259901 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c269e04-3679-4f6b-9ddb-2aef3666bde9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:46:18.261314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1c269e04-3679-4f6b-9ddb-2aef3666bde9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:46:18.329660 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c269e04-3679-4f6b-9ddb-2aef3666bde9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:46:18.333318 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80744af-a36a-4779-8f83-6b4270b40170 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:46:18.333926 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80744af-a36a-4779-8f83-6b4270b40170 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:46:18.334205 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80744af-a36a-4779-8f83-6b4270b40170 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:46:18.334476 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80744af-a36a-4779-8f83-6b4270b40170 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:46:18.343304 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80744af-a36a-4779-8f83-6b4270b40170 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:46:18.344723 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80744af-a36a-4779-8f83-6b4270b40170 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:46:18.354551 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1287/2580] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:46:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:46:18.749643 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92853da4-1657-4633-9aac-47d80571417f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:46:18.751756 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-92853da4-1657-4633-9aac-47d80571417f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:46:18.763337 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92853da4-1657-4633-9aac-47d80571417f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:46:18.768272 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b933e451-007a-440a-82e2-ad04df0f53b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:46:18.768968 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933e451-007a-440a-82e2-ad04df0f53b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:46:18.769288 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933e451-007a-440a-82e2-ad04df0f53b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:46:18.769570 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933e451-007a-440a-82e2-ad04df0f53b4 admin admin] PATH_INFO: `/v3/projects/3b21d4d29aae43889d1904dbca72c88f` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:46:18.770757 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b933e451-007a-440a-82e2-ad04df0f53b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b21d4d29aae43889d1904dbca72c88f)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:46:18.772665 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b933e451-007a-440a-82e2-ad04df0f53b4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:46:18.897033 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b933e451-007a-440a-82e2-ad04df0f53b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b21d4d29aae43889d1904dbca72c88f', 'request_id': 'req-b933e451-007a-440a-82e2-ad04df0f53b4'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:46:18.947736 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b933e451-007a-440a-82e2-ad04df0f53b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b21d4d29aae43889d1904dbca72c88f', 'request_id': 'req-b933e451-007a-440a-82e2-ad04df0f53b4'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:46:18.953402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b933e451-007a-440a-82e2-ad04df0f53b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b21d4d29aae43889d1904dbca72c88f', 'request_id': 'req-b933e451-007a-440a-82e2-ad04df0f53b4'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:46:18.964917 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b933e451-007a-440a-82e2-ad04df0f53b4 admin admin] The token cache is being invalidate because project 3b21d4d29aae43889d1904dbca72c88f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:46:18.965201 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b933e451-007a-440a-82e2-ad04df0f53b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:46:18.965828 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b933e451-007a-440a-82e2-ad04df0f53b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:46:18.967357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1294/2581] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:46:18 2022] DELETE /identity/v3/projects/3b21d4d29aae43889d1904dbca72c88f => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:46:18.996827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8246da7-8d17-4d80-8ddd-ab96f2af9550 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:46:18.997327 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8246da7-8d17-4d80-8ddd-ab96f2af9550 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:46:18.997428 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8246da7-8d17-4d80-8ddd-ab96f2af9550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:46:19.016956 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-a8246da7-8d17-4d80-8ddd-ab96f2af9550 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:46:19.018165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a8246da7-8d17-4d80-8ddd-ab96f2af9550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:46:19.023947 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:46:19.080349 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1288/2582] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:46:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:46:19.088744 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327781cc-41af-40c4-8e81-fcbf68adc53c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:46:19.089035 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327781cc-41af-40c4-8e81-fcbf68adc53c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:46:19.089316 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327781cc-41af-40c4-8e81-fcbf68adc53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:46:19.109110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-327781cc-41af-40c4-8e81-fcbf68adc53c None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:46:19.110318 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-327781cc-41af-40c4-8e81-fcbf68adc53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:46:19.118086 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1295/2583] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:46:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:46:19.122151 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8246da7-8d17-4d80-8ddd-ab96f2af9550 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:46:19.131753 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8246da7-8d17-4d80-8ddd-ab96f2af9550 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:46:19.136222 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ada569-10e3-4d45-9869-6d79af1750b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:46:19.136825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ada569-10e3-4d45-9869-6d79af1750b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:46:19.137103 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ada569-10e3-4d45-9869-6d79af1750b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:46:19.137376 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ada569-10e3-4d45-9869-6d79af1750b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:46:19.138017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ada569-10e3-4d45-9869-6d79af1750b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 09:46:19.138319 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ada569-10e3-4d45-9869-6d79af1750b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:46:19.139934 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ada569-10e3-4d45-9869-6d79af1750b6 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:46:19.152300 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1289/2584] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 09:46:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 09:46:19.156267 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327781cc-41af-40c4-8e81-fcbf68adc53c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:46:19.165907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327781cc-41af-40c4-8e81-fcbf68adc53c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:46:19.170526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e24b6-cf22-4db1-a9a6-0f5d352a99c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:46:19.171159 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e24b6-cf22-4db1-a9a6-0f5d352a99c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:46:19.171441 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e24b6-cf22-4db1-a9a6-0f5d352a99c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:46:19.171714 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e24b6-cf22-4db1-a9a6-0f5d352a99c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:46:19.172351 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362e24b6-cf22-4db1-a9a6-0f5d352a99c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:46:19.174234 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362e24b6-cf22-4db1-a9a6-0f5d352a99c8 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:46:19.268901 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1296/2585] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:46:19 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:46:19.273418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ada569-10e3-4d45-9869-6d79af1750b6 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:46:19.283839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ada569-10e3-4d45-9869-6d79af1750b6 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:46:19.339629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ab070-8006-4c3c-9ede-267c0a2b3569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:46:19.340231 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ab070-8006-4c3c-9ede-267c0a2b3569 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:46:19.340513 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ab070-8006-4c3c-9ede-267c0a2b3569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:46:19.340788 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ab070-8006-4c3c-9ede-267c0a2b3569 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:46:19.341426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781ab070-8006-4c3c-9ede-267c0a2b3569 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:46:19.343358 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781ab070-8006-4c3c-9ede-267c0a2b3569 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:46:19.405721 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1290/2586] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:46:19 2022] POST /identity/v3/users => generated 292 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 09:46:19.409689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e24b6-cf22-4db1-a9a6-0f5d352a99c8 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:46:19.419889 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e24b6-cf22-4db1-a9a6-0f5d352a99c8 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:46:19.424504 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343e0e9-1df3-4753-bfe0-7dc3f32c4194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:46:19.425098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343e0e9-1df3-4753-bfe0-7dc3f32c4194 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:46:19.425382 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343e0e9-1df3-4753-bfe0-7dc3f32c4194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:46:19.425658 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343e0e9-1df3-4753-bfe0-7dc3f32c4194 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:46:19.426299 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b343e0e9-1df3-4753-bfe0-7dc3f32c4194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:46:19.427911 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b343e0e9-1df3-4753-bfe0-7dc3f32c4194 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:46:19.437677 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1297/2587] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:46:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:46:19.441332 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ab070-8006-4c3c-9ede-267c0a2b3569 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:46:19.451710 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ab070-8006-4c3c-9ede-267c0a2b3569 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:46:19.456378 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd7bb61-0942-411b-90a3-2420df2910a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:46:19.456971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7bb61-0942-411b-90a3-2420df2910a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:46:19.457252 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7bb61-0942-411b-90a3-2420df2910a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:46:19.457525 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd7bb61-0942-411b-90a3-2420df2910a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:46:19.458163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd7bb61-0942-411b-90a3-2420df2910a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:46:19.459759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd7bb61-0942-411b-90a3-2420df2910a2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:46:19.470875 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1291/2588] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:46:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:46:19.474565 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343e0e9-1df3-4753-bfe0-7dc3f32c4194 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:46:19.484184 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343e0e9-1df3-4753-bfe0-7dc3f32c4194 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:46:19.489064 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523e099-3536-4379-8a85-673c308d0140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:46:19.489705 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523e099-3536-4379-8a85-673c308d0140 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:46:19.489985 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523e099-3536-4379-8a85-673c308d0140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:46:19.490255 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523e099-3536-4379-8a85-673c308d0140 admin admin] PATH_INFO: `/v3/projects/6939595a200a438facc915ebdb3c3679/users/87691a170289480dbc1e283834d9e400/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:46:19.505464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523e099-3536-4379-8a85-673c308d0140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6939595a200a438facc915ebdb3c3679, user_id=87691a170289480dbc1e283834d9e400, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:46:19.508616 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523e099-3536-4379-8a85-673c308d0140 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:46:19.552885 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1298/2589] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:46:19 2022] PUT /identity/v3/projects/6939595a200a438facc915ebdb3c3679/users/87691a170289480dbc1e283834d9e400/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:46:19.558371 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b561e55-9f7f-47db-be03-94ec71395f11 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:46:19.558653 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b561e55-9f7f-47db-be03-94ec71395f11 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:46:19.558955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b561e55-9f7f-47db-be03-94ec71395f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:46:19.577856 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0b561e55-9f7f-47db-be03-94ec71395f11 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:46:19.579081 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0b561e55-9f7f-47db-be03-94ec71395f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:46:19.639626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1292/2590] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:46:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:46:19.649625 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523e099-3536-4379-8a85-673c308d0140 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:46:19.651590 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7523e099-3536-4379-8a85-673c308d0140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:46:19.673397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:46:19.707710 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523e099-3536-4379-8a85-673c308d0140 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:46:19.711861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e10e8c-4b61-4624-9744-061a1750efb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:46:19.712484 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e10e8c-4b61-4624-9744-061a1750efb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:46:19.712761 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e10e8c-4b61-4624-9744-061a1750efb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:46:19.713033 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e10e8c-4b61-4624-9744-061a1750efb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:46:19.721955 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e10e8c-4b61-4624-9744-061a1750efb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:46:19.724716 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e10e8c-4b61-4624-9744-061a1750efb5 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:46:19.733055 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1299/2591] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:46:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:46:24.756983 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbec02b-f255-4554-aadd-2ca818233b85 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:46:24.757539 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbec02b-f255-4554-aadd-2ca818233b85 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:46:24.757652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbec02b-f255-4554-aadd-2ca818233b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:46:24.788097 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ffbec02b-f255-4554-aadd-2ca818233b85 None None] MFA Rules not processed for user `87691a170289480dbc1e283834d9e400`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:46:24.789815 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ffbec02b-f255-4554-aadd-2ca818233b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:46:24.795797 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:46:24.861591 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1293/2592] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:46:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:46:24.866046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e10e8c-4b61-4624-9744-061a1750efb5 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:46:24.876388 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e10e8c-4b61-4624-9744-061a1750efb5 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:46:24.881187 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a112da0-d785-497a-8ae4-38568ea86fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:46:24.881816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a112da0-d785-497a-8ae4-38568ea86fcb admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:46:24.882122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a112da0-d785-497a-8ae4-38568ea86fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:46:24.883086 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a112da0-d785-497a-8ae4-38568ea86fcb admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:46:24.883224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a112da0-d785-497a-8ae4-38568ea86fcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:46:24.885101 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a112da0-d785-497a-8ae4-38568ea86fcb admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:46:25.001560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1300/2593] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 09:46:24 2022] POST /identity/v3/projects => generated 381 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:46:25.005683 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbec02b-f255-4554-aadd-2ca818233b85 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:46:25.016602 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbec02b-f255-4554-aadd-2ca818233b85 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:46:25.072548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a5cd1-8bd1-4baf-9cfb-5fe211b02a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:46:25.073157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a5cd1-8bd1-4baf-9cfb-5fe211b02a89 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:46:25.073436 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a5cd1-8bd1-4baf-9cfb-5fe211b02a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:46:25.073712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9a5cd1-8bd1-4baf-9cfb-5fe211b02a89 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:46:25.074348 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9a5cd1-8bd1-4baf-9cfb-5fe211b02a89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:46:25.076298 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9a5cd1-8bd1-4baf-9cfb-5fe211b02a89 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:46:25.141963 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1294/2594] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 09:46:25 2022] POST /identity/v3/users => generated 297 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 09:46:25.146051 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a112da0-d785-497a-8ae4-38568ea86fcb admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:46:25.157204 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a112da0-d785-497a-8ae4-38568ea86fcb admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:46:25.161796 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b1921-708c-4129-ab3e-d061cfd12427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:46:25.162391 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b1921-708c-4129-ab3e-d061cfd12427 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:46:25.162673 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b1921-708c-4129-ab3e-d061cfd12427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:46:25.162980 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b1921-708c-4129-ab3e-d061cfd12427 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:46:25.163643 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b1921-708c-4129-ab3e-d061cfd12427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:46:25.165817 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b1921-708c-4129-ab3e-d061cfd12427 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:46:25.176123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1301/2595] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:46:25.180154 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a5cd1-8bd1-4baf-9cfb-5fe211b02a89 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:46:25.190596 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9a5cd1-8bd1-4baf-9cfb-5fe211b02a89 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:46:25.195802 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6159ff-6a8f-4d01-8d21-481813a3facf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:46:25.196469 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6159ff-6a8f-4d01-8d21-481813a3facf admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:46:25.196751 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6159ff-6a8f-4d01-8d21-481813a3facf admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:46:25.197026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6159ff-6a8f-4d01-8d21-481813a3facf admin admin] PATH_INFO: `/v3/projects/7f3d13a1f45e4210aecc60afc046c31a/users/fcc231295a1a4065a6008e6cabf89466/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:46:25.213202 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6159ff-6a8f-4d01-8d21-481813a3facf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f3d13a1f45e4210aecc60afc046c31a, user_id=fcc231295a1a4065a6008e6cabf89466, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:46:25.215883 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6159ff-6a8f-4d01-8d21-481813a3facf admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:46:25.337961 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1295/2596] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:46:25 2022] PUT /identity/v3/projects/7f3d13a1f45e4210aecc60afc046c31a/users/fcc231295a1a4065a6008e6cabf89466/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:46:25.342127 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b1921-708c-4129-ab3e-d061cfd12427 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:46:25.352371 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b1921-708c-4129-ab3e-d061cfd12427 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:46:25.409361 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ead279-bbb1-4f7a-bd36-5be795e1d24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:46:25.409972 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ead279-bbb1-4f7a-bd36-5be795e1d24f admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:46:25.410256 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ead279-bbb1-4f7a-bd36-5be795e1d24f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:46:25.410529 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ead279-bbb1-4f7a-bd36-5be795e1d24f admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:46:25.411209 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ead279-bbb1-4f7a-bd36-5be795e1d24f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:46:25.412872 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ead279-bbb1-4f7a-bd36-5be795e1d24f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:46:25.422941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1302/2597] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 09:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 09:46:25.427232 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6159ff-6a8f-4d01-8d21-481813a3facf admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:46:25.438875 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6159ff-6a8f-4d01-8d21-481813a3facf admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:46:25.443142 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392dbc20-92a1-4a76-829d-512056d42ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:46:25.443798 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392dbc20-92a1-4a76-829d-512056d42ec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:46:25.444100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392dbc20-92a1-4a76-829d-512056d42ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:46:25.444387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392dbc20-92a1-4a76-829d-512056d42ec7 admin admin] PATH_INFO: `/v3/projects/7f3d13a1f45e4210aecc60afc046c31a/users/fcc231295a1a4065a6008e6cabf89466/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:46:25.446834 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392dbc20-92a1-4a76-829d-512056d42ec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f3d13a1f45e4210aecc60afc046c31a, user_id=fcc231295a1a4065a6008e6cabf89466, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:46:25.449441 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392dbc20-92a1-4a76-829d-512056d42ec7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:46:25.461358 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1296/2598] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 09:46:25 2022] PUT /identity/v3/projects/7f3d13a1f45e4210aecc60afc046c31a/users/fcc231295a1a4065a6008e6cabf89466/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:46:30.767356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070873e2-0c87-4d19-8ba8-79206851af81 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:46:30.767861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070873e2-0c87-4d19-8ba8-79206851af81 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:46:30.767980 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070873e2-0c87-4d19-8ba8-79206851af81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:46:30.802588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-070873e2-0c87-4d19-8ba8-79206851af81 None None] MFA Rules not processed for user `fcc231295a1a4065a6008e6cabf89466`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:46:30.803900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-070873e2-0c87-4d19-8ba8-79206851af81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:46:30.883236 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1303/2599] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 09:46:30 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:46:30.896020 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392dbc20-92a1-4a76-829d-512056d42ec7 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:46:30.897767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-392dbc20-92a1-4a76-829d-512056d42ec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:46:30.966871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:46:30.980839 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392dbc20-92a1-4a76-829d-512056d42ec7 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:46:30.984552 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5da857-a28e-4a62-8c2a-414e2a86b391 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b555bb94ce44ce918cbf2a012576a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:46:30.985162 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5da857-a28e-4a62-8c2a-414e2a86b391 service glance] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:46:30.985442 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5da857-a28e-4a62-8c2a-414e2a86b391 service glance] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:46:30.985714 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5da857-a28e-4a62-8c2a-414e2a86b391 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:46:31.045397 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5da857-a28e-4a62-8c2a-414e2a86b391 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:46:31.046864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5da857-a28e-4a62-8c2a-414e2a86b391 service glance] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:46:31.056157 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1297/2600] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:46:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:46:31.255637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2941da2-f7da-491d-985d-62d26e5d20c7 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:46:31.255926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2941da2-f7da-491d-985d-62d26e5d20c7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:46:31.256203 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2941da2-f7da-491d-985d-62d26e5d20c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:46:31.275001 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d2941da2-f7da-491d-985d-62d26e5d20c7 None None] MFA Rules not processed for user `5233689a53364258b86ad2fa012d8db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:46:31.276461 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d2941da2-f7da-491d-985d-62d26e5d20c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:46:31.281625 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:46:31.332216 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1304/2601] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Sep 2 09:46:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:46:31.455036 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5da857-a28e-4a62-8c2a-414e2a86b391 service glance] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:46:31.456608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7a5da857-a28e-4a62-8c2a-414e2a86b391 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:46:31.550812 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5da857-a28e-4a62-8c2a-414e2a86b391 service glance] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:46:31.556800 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdb589-54c3-42d2-8fa7-11872231a6d0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30fe233a7d4c4fbea6ede233bd9ff8c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:46:31.557423 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdb589-54c3-42d2-8fa7-11872231a6d0 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:46:31.557705 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdb589-54c3-42d2-8fa7-11872231a6d0 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:46:31.557980 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcdb589-54c3-42d2-8fa7-11872231a6d0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:46:31.567283 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcdb589-54c3-42d2-8fa7-11872231a6d0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:46:31.568862 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcdb589-54c3-42d2-8fa7-11872231a6d0 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:46:31.577908 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1298/2602] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Sep 2 09:46:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 09:46:41.348176 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=20 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:46:41.574699 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=67 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:46:44.728409 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2941da2-f7da-491d-985d-62d26e5d20c7 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:46:44.729885 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d2941da2-f7da-491d-985d-62d26e5d20c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:46:44.801654 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2941da2-f7da-491d-985d-62d26e5d20c7 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:46:44.805247 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320b259-c64b-46a2-982c-4cef102b0f7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:46:44.806397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320b259-c64b-46a2-982c-4cef102b0f7e service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:46:44.806675 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320b259-c64b-46a2-982c-4cef102b0f7e service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:46:44.806971 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320b259-c64b-46a2-982c-4cef102b0f7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:46:44.808739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7320b259-c64b-46a2-982c-4cef102b0f7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:46:44.822164 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-7320b259-c64b-46a2-982c-4cef102b0f7e service cinder] Could not find project: 3b21d4d29aae43889d1904dbca72c88f.: keystone.exception.ProjectNotFound: Could not find project: 3b21d4d29aae43889d1904dbca72c88f. Sep 02 09:46:44.822164 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 02 09:46:44.822164 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 02 09:46:44.822164 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 02 09:46:44.822164 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 02 09:46:44.822164 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 02 09:46:44.822164 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 02 09:46:44.822164 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 02 09:46:44.822164 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 02 09:46:44.822164 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 02 09:46:44.822164 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 02 09:46:44.822164 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 02 09:46:44.822164 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 02 09:46:44.822164 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 02 09:46:44.822164 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 02 09:46:44.823496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 02 09:46:44.823496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 02 09:46:44.823496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 02 09:46:44.823496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:46:44.823496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:46:44.823496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Sep 02 09:46:44.823496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Sep 02 09:46:44.823496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:46:44.823496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:46:44.823496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:46:44.823496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:46:44.823496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:46:44.823496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:46:44.823496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:46:44.823496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:46:44.823496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:46:44.823496 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:46:44.827039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:46:44.827039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:46:44.827039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:46:44.827039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:46:44.827039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Sep 02 09:46:44.827039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Sep 02 09:46:44.827039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Sep 02 09:46:44.827039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._validate_token_resources() Sep 02 09:46:44.827039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Sep 02 09:46:44.827039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Sep 02 09:46:44.827039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Sep 02 09:46:44.827039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Sep 02 09:46:44.827039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 02 09:46:44.827039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 02 09:46:44.827039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 02 09:46:44.827039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 02 09:46:44.827039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 02 09:46:44.828347 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Sep 02 09:46:44.828347 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 02 09:46:44.828347 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Sep 02 09:46:44.828347 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 02 09:46:44.828347 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Sep 02 09:46:44.828347 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 02 09:46:44.828347 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 02 09:46:44.828347 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 02 09:46:44.828347 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Sep 02 09:46:44.828347 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 02 09:46:44.828347 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Sep 02 09:46:44.828347 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 02 09:46:44.828347 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 02 09:46:44.828347 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 02 09:46:44.828347 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 02 09:46:44.828347 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 02 09:46:44.828347 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 02 09:46:44.829664 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 3b21d4d29aae43889d1904dbca72c88f. Sep 02 09:46:44.829664 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Sep 02 09:46:44.829664 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1305/2603] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:46:44 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 96 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 02 09:46:47.251243 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdb589-54c3-42d2-8fa7-11872231a6d0 service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:46:47.254111 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-afcdb589-54c3-42d2-8fa7-11872231a6d0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:46:47.321938 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcdb589-54c3-42d2-8fa7-11872231a6d0 service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:46:47.326963 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b66f14-1cbd-478f-8ae8-0eed9fe7d8df service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5177fa4d168c4c219e09c7178bbbc790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:46:47.327560 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b66f14-1cbd-478f-8ae8-0eed9fe7d8df service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:46:47.327849 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b66f14-1cbd-478f-8ae8-0eed9fe7d8df service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:46:47.328108 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b66f14-1cbd-478f-8ae8-0eed9fe7d8df service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:46:47.329799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-42b66f14-1cbd-478f-8ae8-0eed9fe7d8df service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:46:47.341812 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b66f14-1cbd-478f-8ae8-0eed9fe7d8df service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:46:47.343400 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b66f14-1cbd-478f-8ae8-0eed9fe7d8df service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:46:47.353933 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1299/2604] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:46:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:46:54.812356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=29 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:46:57.349566 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=32 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:47:41.968980 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2650a3-273d-43e0-b14b-bb64902e42a7 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:47:41.969483 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2650a3-273d-43e0-b14b-bb64902e42a7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:47:41.969607 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2650a3-273d-43e0-b14b-bb64902e42a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:47:41.989693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-9f2650a3-273d-43e0-b14b-bb64902e42a7 None None] MFA Rules not processed for user `5233689a53364258b86ad2fa012d8db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 09:47:41.990947 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9f2650a3-273d-43e0-b14b-bb64902e42a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:47:41.998042 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1306/2605] 10.222.0.57 () {58 vars in 967 bytes} [Fri Sep 2 09:47:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 09:47:42.139697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b66f14-1cbd-478f-8ae8-0eed9fe7d8df service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:47:42.151784 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b66f14-1cbd-478f-8ae8-0eed9fe7d8df service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:47:42.156739 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361e236-fc17-41d8-a4d2-785cd15a3008 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30fe233a7d4c4fbea6ede233bd9ff8c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:47:42.157336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361e236-fc17-41d8-a4d2-785cd15a3008 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:47:42.158265 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361e236-fc17-41d8-a4d2-785cd15a3008 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:47:42.158537 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361e236-fc17-41d8-a4d2-785cd15a3008 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:47:42.166398 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1361e236-fc17-41d8-a4d2-785cd15a3008 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:47:42.168010 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1361e236-fc17-41d8-a4d2-785cd15a3008 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:47:42.175846 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1300/2606] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Sep 2 09:47:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 02 09:47:46.467968 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2650a3-273d-43e0-b14b-bb64902e42a7 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:47:46.479907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2650a3-273d-43e0-b14b-bb64902e42a7 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:47:46.484857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4968df2-5c5c-4c3d-b1c7-0dfb1f62bf83 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5177fa4d168c4c219e09c7178bbbc790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:47:46.485546 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4968df2-5c5c-4c3d-b1c7-0dfb1f62bf83 service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:47:46.485908 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4968df2-5c5c-4c3d-b1c7-0dfb1f62bf83 service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:47:46.486251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4968df2-5c5c-4c3d-b1c7-0dfb1f62bf83 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:47:46.488087 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b4968df2-5c5c-4c3d-b1c7-0dfb1f62bf83 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:47:46.499693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4968df2-5c5c-4c3d-b1c7-0dfb1f62bf83 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:47:46.501860 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4968df2-5c5c-4c3d-b1c7-0dfb1f62bf83 service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:47:46.512173 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1307/2607] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:47:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:47:50.409437 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361e236-fc17-41d8-a4d2-785cd15a3008 service swift] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:47:50.476226 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361e236-fc17-41d8-a4d2-785cd15a3008 service swift] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:47:50.484138 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c7879-9d3b-4e74-86de-bf67174c5891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:47:50.484138 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c7879-9d3b-4e74-86de-bf67174c5891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:47:50.484138 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c7879-9d3b-4e74-86de-bf67174c5891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:47:50.484138 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c7879-9d3b-4e74-86de-bf67174c5891 admin admin] PATH_INFO: `/v3/users/87691a170289480dbc1e283834d9e400` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:47:50.484596 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c7879-9d3b-4e74-86de-bf67174c5891 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87691a170289480dbc1e283834d9e400)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:47:50.485007 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c7879-9d3b-4e74-86de-bf67174c5891 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:47:50.648022 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-323c7879-9d3b-4e74-86de-bf67174c5891 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87691a170289480dbc1e283834d9e400'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:47:50.713600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-323c7879-9d3b-4e74-86de-bf67174c5891 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87691a170289480dbc1e283834d9e400'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:47:50.714239 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-323c7879-9d3b-4e74-86de-bf67174c5891 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87691a170289480dbc1e283834d9e400'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:47:50.714973 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-323c7879-9d3b-4e74-86de-bf67174c5891 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87691a170289480dbc1e283834d9e400'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:47:50.728412 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-323c7879-9d3b-4e74-86de-bf67174c5891 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87691a170289480dbc1e283834d9e400'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:47:50.741364 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1301/2608] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:47:50 2022] DELETE /identity/v3/users/87691a170289480dbc1e283834d9e400 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:47:50.994407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4968df2-5c5c-4c3d-b1c7-0dfb1f62bf83 service placement] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:47:51.006396 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4968df2-5c5c-4c3d-b1c7-0dfb1f62bf83 service placement] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:47:51.063393 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d75d3-2b9f-4834-94ef-186f771f6e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:47:51.064076 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d75d3-2b9f-4834-94ef-186f771f6e90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:47:51.064360 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d75d3-2b9f-4834-94ef-186f771f6e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:47:51.066497 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d75d3-2b9f-4834-94ef-186f771f6e90 admin admin] PATH_INFO: `/v3/projects/6939595a200a438facc915ebdb3c3679` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:47:51.067779 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d75d3-2b9f-4834-94ef-186f771f6e90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6939595a200a438facc915ebdb3c3679)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:47:51.069710 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215d75d3-2b9f-4834-94ef-186f771f6e90 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:47:51.178518 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-215d75d3-2b9f-4834-94ef-186f771f6e90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6939595a200a438facc915ebdb3c3679', 'request_id': 'req-215d75d3-2b9f-4834-94ef-186f771f6e90'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:47:51.253835 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-215d75d3-2b9f-4834-94ef-186f771f6e90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6939595a200a438facc915ebdb3c3679', 'request_id': 'req-215d75d3-2b9f-4834-94ef-186f771f6e90'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:47:51.259521 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-215d75d3-2b9f-4834-94ef-186f771f6e90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6939595a200a438facc915ebdb3c3679', 'request_id': 'req-215d75d3-2b9f-4834-94ef-186f771f6e90'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:47:51.270958 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-215d75d3-2b9f-4834-94ef-186f771f6e90 admin admin] The token cache is being invalidate because project 6939595a200a438facc915ebdb3c3679 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:47:51.271242 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-215d75d3-2b9f-4834-94ef-186f771f6e90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:47:51.271867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-215d75d3-2b9f-4834-94ef-186f771f6e90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:47:51.273339 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1308/2609] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:47:50 2022] DELETE /identity/v3/projects/6939595a200a438facc915ebdb3c3679 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:47:51.277762 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c7879-9d3b-4e74-86de-bf67174c5891 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:47:51.279185 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-323c7879-9d3b-4e74-86de-bf67174c5891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:47:51.293038 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c7879-9d3b-4e74-86de-bf67174c5891 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:47:51.347430 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d500f157-fd59-48ae-be2b-8327a43e2329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:47:51.348047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500f157-fd59-48ae-be2b-8327a43e2329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:47:51.348321 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500f157-fd59-48ae-be2b-8327a43e2329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:47:51.348594 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d500f157-fd59-48ae-be2b-8327a43e2329 admin admin] PATH_INFO: `/v3/users/fcc231295a1a4065a6008e6cabf89466` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:47:51.349702 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d500f157-fd59-48ae-be2b-8327a43e2329 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcc231295a1a4065a6008e6cabf89466)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:47:51.351650 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d500f157-fd59-48ae-be2b-8327a43e2329 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:47:51.356776 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:47:51.641613 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d500f157-fd59-48ae-be2b-8327a43e2329 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcc231295a1a4065a6008e6cabf89466'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:47:51.770060 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d500f157-fd59-48ae-be2b-8327a43e2329 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcc231295a1a4065a6008e6cabf89466'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:47:51.770611 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d500f157-fd59-48ae-be2b-8327a43e2329 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcc231295a1a4065a6008e6cabf89466'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:47:51.771263 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d500f157-fd59-48ae-be2b-8327a43e2329 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcc231295a1a4065a6008e6cabf89466'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:47:51.785752 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d500f157-fd59-48ae-be2b-8327a43e2329 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcc231295a1a4065a6008e6cabf89466'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:47:51.798377 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1302/2610] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:47:51 2022] DELETE /identity/v3/users/fcc231295a1a4065a6008e6cabf89466 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:47:52.044133 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d75d3-2b9f-4834-94ef-186f771f6e90 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:47:52.047906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-215d75d3-2b9f-4834-94ef-186f771f6e90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:47:52.059472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d75d3-2b9f-4834-94ef-186f771f6e90 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:47:52.116016 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae52b1-fbd4-48dd-ad4d-88c4a6e89c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:47:52.116637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae52b1-fbd4-48dd-ad4d-88c4a6e89c07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:47:52.116914 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae52b1-fbd4-48dd-ad4d-88c4a6e89c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:47:52.117186 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae52b1-fbd4-48dd-ad4d-88c4a6e89c07 admin admin] PATH_INFO: `/v3/projects/7f3d13a1f45e4210aecc60afc046c31a` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:47:52.118276 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae52b1-fbd4-48dd-ad4d-88c4a6e89c07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f3d13a1f45e4210aecc60afc046c31a)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:47:52.120198 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae52b1-fbd4-48dd-ad4d-88c4a6e89c07 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:47:52.131841 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:47:52.249873 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-47ae52b1-fbd4-48dd-ad4d-88c4a6e89c07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f3d13a1f45e4210aecc60afc046c31a', 'request_id': 'req-47ae52b1-fbd4-48dd-ad4d-88c4a6e89c07'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:47:52.341573 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-47ae52b1-fbd4-48dd-ad4d-88c4a6e89c07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f3d13a1f45e4210aecc60afc046c31a', 'request_id': 'req-47ae52b1-fbd4-48dd-ad4d-88c4a6e89c07'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:47:52.347904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-47ae52b1-fbd4-48dd-ad4d-88c4a6e89c07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f3d13a1f45e4210aecc60afc046c31a', 'request_id': 'req-47ae52b1-fbd4-48dd-ad4d-88c4a6e89c07'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:47:52.360661 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-47ae52b1-fbd4-48dd-ad4d-88c4a6e89c07 admin admin] The token cache is being invalidate because project 7f3d13a1f45e4210aecc60afc046c31a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:47:52.360948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-47ae52b1-fbd4-48dd-ad4d-88c4a6e89c07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:47:52.362150 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-47ae52b1-fbd4-48dd-ad4d-88c4a6e89c07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:47:52.363711 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1309/2611] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:47:52 2022] DELETE /identity/v3/projects/7f3d13a1f45e4210aecc60afc046c31a => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:48:01.794727 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:48:02.357525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=19,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:48:44.900387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d500f157-fd59-48ae-be2b-8327a43e2329 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:48:44.901800 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d500f157-fd59-48ae-be2b-8327a43e2329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:48:44.960088 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d500f157-fd59-48ae-be2b-8327a43e2329 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:48:45.008536 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd63bb3d-ee51-4e5b-aa71-9d95f2010cf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:48:45.009142 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63bb3d-ee51-4e5b-aa71-9d95f2010cf0 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:48:45.009419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63bb3d-ee51-4e5b-aa71-9d95f2010cf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:48:45.009690 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63bb3d-ee51-4e5b-aa71-9d95f2010cf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:48:45.011468 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-cd63bb3d-ee51-4e5b-aa71-9d95f2010cf0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:48:45.093695 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd63bb3d-ee51-4e5b-aa71-9d95f2010cf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:48:45.095725 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd63bb3d-ee51-4e5b-aa71-9d95f2010cf0 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:48:45.106221 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1303/2612] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:48:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:48:45.114059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae52b1-fbd4-48dd-ad4d-88c4a6e89c07 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:48:45.115477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-47ae52b1-fbd4-48dd-ad4d-88c4a6e89c07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:48:45.127457 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae52b1-fbd4-48dd-ad4d-88c4a6e89c07 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:48:45.131009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b3349-75ea-4f95-8751-72e7b91f4433 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:48:45.131605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b3349-75ea-4f95-8751-72e7b91f4433 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:48:45.131885 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b3349-75ea-4f95-8751-72e7b91f4433 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:48:45.132157 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b3349-75ea-4f95-8751-72e7b91f4433 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:48:45.134006 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-825b3349-75ea-4f95-8751-72e7b91f4433 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:48:45.203248 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b3349-75ea-4f95-8751-72e7b91f4433 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:48:45.205195 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b3349-75ea-4f95-8751-72e7b91f4433 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:48:45.214047 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1310/2613] 10.222.0.57 () {60 vars in 1396 bytes} [Fri Sep 2 09:48:45 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3270 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:48:45.256624 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd63bb3d-ee51-4e5b-aa71-9d95f2010cf0 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:48:45.267848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd63bb3d-ee51-4e5b-aa71-9d95f2010cf0 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:48:45.271729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad1519-c969-40d9-85b6-063a4daee00c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:48:45.272327 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad1519-c969-40d9-85b6-063a4daee00c service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:48:45.272608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad1519-c969-40d9-85b6-063a4daee00c service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:48:45.272881 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad1519-c969-40d9-85b6-063a4daee00c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:48:45.274915 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4fad1519-c969-40d9-85b6-063a4daee00c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:48:45.289434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fad1519-c969-40d9-85b6-063a4daee00c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:48:45.290539 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fad1519-c969-40d9-85b6-063a4daee00c service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:48:45.302551 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1304/2614] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:48:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:48:45.309895 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b3349-75ea-4f95-8751-72e7b91f4433 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:48:45.318847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b3349-75ea-4f95-8751-72e7b91f4433 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:48:45.322488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a60d361-f1f3-4a08-92b1-a1c918a4ec88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:48:45.323108 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60d361-f1f3-4a08-92b1-a1c918a4ec88 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:48:45.323385 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60d361-f1f3-4a08-92b1-a1c918a4ec88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:48:45.323654 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60d361-f1f3-4a08-92b1-a1c918a4ec88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:48:45.325563 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6a60d361-f1f3-4a08-92b1-a1c918a4ec88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:48:45.411600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a60d361-f1f3-4a08-92b1-a1c918a4ec88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:48:45.413030 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a60d361-f1f3-4a08-92b1-a1c918a4ec88 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:48:45.433275 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1311/2615] 10.222.0.57 () {60 vars in 1396 bytes} [Fri Sep 2 09:48:45 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:48:51.891468 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad1519-c969-40d9-85b6-063a4daee00c service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:48:51.892935 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4fad1519-c969-40d9-85b6-063a4daee00c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:48:52.049610 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:48:52.071234 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad1519-c969-40d9-85b6-063a4daee00c service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:48:52.074834 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4eb5b-868d-46fa-b309-18a2cb799230 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:48:52.075427 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4eb5b-868d-46fa-b309-18a2cb799230 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:48:52.075701 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4eb5b-868d-46fa-b309-18a2cb799230 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:48:52.075967 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4eb5b-868d-46fa-b309-18a2cb799230 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:48:52.077743 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3cb4eb5b-868d-46fa-b309-18a2cb799230 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:48:52.087607 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4eb5b-868d-46fa-b309-18a2cb799230 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:48:52.089086 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4eb5b-868d-46fa-b309-18a2cb799230 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:48:52.098446 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1305/2616] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:48:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:48:55.429273 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=74 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:49:02.094040 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=11 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:49:44.935419 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a60d361-f1f3-4a08-92b1-a1c918a4ec88 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:49:44.936881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6a60d361-f1f3-4a08-92b1-a1c918a4ec88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:49:44.957185 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a60d361-f1f3-4a08-92b1-a1c918a4ec88 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:49:44.962822 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887d4bb-15bb-4f61-897a-23b47c6259f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:49:44.963460 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887d4bb-15bb-4f61-897a-23b47c6259f2 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:49:44.964422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887d4bb-15bb-4f61-897a-23b47c6259f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:49:44.964706 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887d4bb-15bb-4f61-897a-23b47c6259f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:49:44.966508 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1887d4bb-15bb-4f61-897a-23b47c6259f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:49:45.038273 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1887d4bb-15bb-4f61-897a-23b47c6259f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:49:45.039886 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1887d4bb-15bb-4f61-897a-23b47c6259f2 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:49:45.048923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1312/2617] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:49:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:49:55.044799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=33 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:50:15.728628 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4eb5b-868d-46fa-b309-18a2cb799230 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:50:15.730056 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3cb4eb5b-868d-46fa-b309-18a2cb799230 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:50:15.741332 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4eb5b-868d-46fa-b309-18a2cb799230 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:50:15.744876 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ab322-0e0e-4312-a479-aef9ccb5fc07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:50:15.745465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ab322-0e0e-4312-a479-aef9ccb5fc07 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:50:15.745742 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ab322-0e0e-4312-a479-aef9ccb5fc07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:50:15.746039 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ab322-0e0e-4312-a479-aef9ccb5fc07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:50:15.747710 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8d0ab322-0e0e-4312-a479-aef9ccb5fc07 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:50:15.816169 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ab322-0e0e-4312-a479-aef9ccb5fc07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:50:15.817532 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ab322-0e0e-4312-a479-aef9ccb5fc07 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:50:15.826299 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1306/2618] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:50:15 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:50:25.822301 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=33 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:51:51.199394 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887d4bb-15bb-4f61-897a-23b47c6259f2 service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:51:51.201470 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1887d4bb-15bb-4f61-897a-23b47c6259f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:51:51.329007 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887d4bb-15bb-4f61-897a-23b47c6259f2 service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:51:51.333998 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ffd90-2c79-4a4a-a03e-f01ace9d379c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5177fa4d168c4c219e09c7178bbbc790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:51:51.334592 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ffd90-2c79-4a4a-a03e-f01ace9d379c service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:51:51.334903 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ffd90-2c79-4a4a-a03e-f01ace9d379c service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:51:51.335177 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ffd90-2c79-4a4a-a03e-f01ace9d379c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:51:51.336877 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6c9ffd90-2c79-4a4a-a03e-f01ace9d379c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:51:51.348731 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ffd90-2c79-4a4a-a03e-f01ace9d379c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:51:51.350307 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ffd90-2c79-4a4a-a03e-f01ace9d379c service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:51:51.360196 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1313/2619] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:51:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:52:01.355790 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=58 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:53:46.011789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ab322-0e0e-4312-a479-aef9ccb5fc07 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:53:46.013111 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8d0ab322-0e0e-4312-a479-aef9ccb5fc07 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:53:46.076186 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ab322-0e0e-4312-a479-aef9ccb5fc07 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:53:46.081688 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d5bcae-37e3-4ca3-8b9a-ea2ce5a61731 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:53:46.082287 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5bcae-37e3-4ca3-8b9a-ea2ce5a61731 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:53:46.082566 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5bcae-37e3-4ca3-8b9a-ea2ce5a61731 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:53:46.082883 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5bcae-37e3-4ca3-8b9a-ea2ce5a61731 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:53:46.090965 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d5bcae-37e3-4ca3-8b9a-ea2ce5a61731 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:53:46.092525 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d5bcae-37e3-4ca3-8b9a-ea2ce5a61731 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:53:46.093375 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ffd90-2c79-4a4a-a03e-f01ace9d379c service placement] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:53:46.120203 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1307/2620] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:53:46 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:53:46.121935 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ffd90-2c79-4a4a-a03e-f01ace9d379c service placement] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:53:46.125481 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e51e7-7cf1-46fd-9550-0506981b64a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:53:46.127908 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e51e7-7cf1-46fd-9550-0506981b64a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:53:46.128191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e51e7-7cf1-46fd-9550-0506981b64a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:53:46.128464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e51e7-7cf1-46fd-9550-0506981b64a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:53:46.137965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e51e7-7cf1-46fd-9550-0506981b64a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:53:46.139508 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e51e7-7cf1-46fd-9550-0506981b64a8 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:53:46.148524 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1314/2621] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:53:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:53:55.895168 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d5bcae-37e3-4ca3-8b9a-ea2ce5a61731 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:53:55.913815 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d5bcae-37e3-4ca3-8b9a-ea2ce5a61731 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:53:55.917379 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99898419-1392-4310-9317-d4bd29a41dd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:53:55.917953 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99898419-1392-4310-9317-d4bd29a41dd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:53:55.918230 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99898419-1392-4310-9317-d4bd29a41dd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:53:55.918500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99898419-1392-4310-9317-d4bd29a41dd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:53:55.925694 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99898419-1392-4310-9317-d4bd29a41dd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:53:55.927149 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99898419-1392-4310-9317-d4bd29a41dd5 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:53:55.935519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1308/2622] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:53:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:53:56.144466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=15 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:54:05.931492 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=45 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:54:44.863848 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e51e7-7cf1-46fd-9550-0506981b64a8 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:54:44.873180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e51e7-7cf1-46fd-9550-0506981b64a8 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:54:44.876992 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55a9b8-5a6f-4ca7-a0ab-c5d5c4d98af0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:54:44.877594 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55a9b8-5a6f-4ca7-a0ab-c5d5c4d98af0 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:54:44.877877 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55a9b8-5a6f-4ca7-a0ab-c5d5c4d98af0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:54:44.878152 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55a9b8-5a6f-4ca7-a0ab-c5d5c4d98af0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:54:44.886846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe55a9b8-5a6f-4ca7-a0ab-c5d5c4d98af0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:54:44.888853 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe55a9b8-5a6f-4ca7-a0ab-c5d5c4d98af0 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:54:44.916631 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1315/2623] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:54:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:54:54.913989 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=13 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:55:14.987783 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99898419-1392-4310-9317-d4bd29a41dd5 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:55:14.996940 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99898419-1392-4310-9317-d4bd29a41dd5 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:55:15.000478 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef66b3-f3f7-4620-9a75-8af12afa8082 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:55:15.001072 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef66b3-f3f7-4620-9a75-8af12afa8082 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:55:15.001346 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef66b3-f3f7-4620-9a75-8af12afa8082 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:55:15.001615 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bef66b3-f3f7-4620-9a75-8af12afa8082 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:55:15.009809 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bef66b3-f3f7-4620-9a75-8af12afa8082 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:55:15.011391 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bef66b3-f3f7-4620-9a75-8af12afa8082 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:55:15.021005 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1309/2624] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:55:14 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:55:17.492075 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55a9b8-5a6f-4ca7-a0ab-c5d5c4d98af0 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:55:17.502592 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55a9b8-5a6f-4ca7-a0ab-c5d5c4d98af0 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:55:17.506371 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850dc488-7e2e-4e7d-81c3-df15fd498d5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:55:17.506974 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850dc488-7e2e-4e7d-81c3-df15fd498d5a service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:55:17.507238 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850dc488-7e2e-4e7d-81c3-df15fd498d5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:55:17.507510 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850dc488-7e2e-4e7d-81c3-df15fd498d5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:55:17.518188 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850dc488-7e2e-4e7d-81c3-df15fd498d5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:55:17.519676 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850dc488-7e2e-4e7d-81c3-df15fd498d5a service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:55:17.530928 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1316/2625] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:55:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:55:18.726399 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef66b3-f3f7-4620-9a75-8af12afa8082 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:55:18.728496 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0bef66b3-f3f7-4620-9a75-8af12afa8082 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:55:18.744709 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bef66b3-f3f7-4620-9a75-8af12afa8082 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:55:18.749064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889d77a-a795-42c4-8a24-66b811c80d29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:55:18.749674 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889d77a-a795-42c4-8a24-66b811c80d29 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:55:18.749951 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889d77a-a795-42c4-8a24-66b811c80d29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:55:18.750221 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889d77a-a795-42c4-8a24-66b811c80d29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:55:18.752128 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3889d77a-a795-42c4-8a24-66b811c80d29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:55:18.838459 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3889d77a-a795-42c4-8a24-66b811c80d29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:55:18.839921 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3889d77a-a795-42c4-8a24-66b811c80d29 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:55:18.859115 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1310/2626] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:55:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:55:19.891043 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850dc488-7e2e-4e7d-81c3-df15fd498d5a service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:55:19.907989 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850dc488-7e2e-4e7d-81c3-df15fd498d5a service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:55:19.909852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6adb57-613b-4267-b28e-c0ca04037814 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:55:19.910190 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6adb57-613b-4267-b28e-c0ca04037814 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:55:19.910544 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6adb57-613b-4267-b28e-c0ca04037814 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:55:19.911086 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6adb57-613b-4267-b28e-c0ca04037814 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:55:19.927063 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6adb57-613b-4267-b28e-c0ca04037814 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:55:19.928606 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6adb57-613b-4267-b28e-c0ca04037814 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:55:19.940244 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1317/2627] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:55:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:55:22.458154 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889d77a-a795-42c4-8a24-66b811c80d29 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:55:22.469871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889d77a-a795-42c4-8a24-66b811c80d29 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:55:22.475080 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4e1fe-6f1a-4021-bde8-b0513a068fd3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5177fa4d168c4c219e09c7178bbbc790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:55:22.475684 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4e1fe-6f1a-4021-bde8-b0513a068fd3 service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:55:22.475981 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4e1fe-6f1a-4021-bde8-b0513a068fd3 service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:55:22.476236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c4e1fe-6f1a-4021-bde8-b0513a068fd3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:55:22.477954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-14c4e1fe-6f1a-4021-bde8-b0513a068fd3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:55:22.490198 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4e1fe-6f1a-4021-bde8-b0513a068fd3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:55:22.491792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c4e1fe-6f1a-4021-bde8-b0513a068fd3 service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:55:22.502968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1311/2628] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:55:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 09:55:26.192596 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6adb57-613b-4267-b28e-c0ca04037814 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:55:26.204851 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6adb57-613b-4267-b28e-c0ca04037814 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:55:26.210579 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936350cc-5062-4d55-9651-c5a0d05cb3f6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5177fa4d168c4c219e09c7178bbbc790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:55:26.211223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936350cc-5062-4d55-9651-c5a0d05cb3f6 service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:55:26.211533 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936350cc-5062-4d55-9651-c5a0d05cb3f6 service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:55:26.211811 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936350cc-5062-4d55-9651-c5a0d05cb3f6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:55:26.213657 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-936350cc-5062-4d55-9651-c5a0d05cb3f6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:55:26.226421 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936350cc-5062-4d55-9651-c5a0d05cb3f6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:55:26.227717 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936350cc-5062-4d55-9651-c5a0d05cb3f6 service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:55:26.239279 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1318/2629] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:55:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:55:26.326284 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4e1fe-6f1a-4021-bde8-b0513a068fd3 service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:55:26.327802 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-14c4e1fe-6f1a-4021-bde8-b0513a068fd3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:55:26.341170 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c4e1fe-6f1a-4021-bde8-b0513a068fd3 service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:55:26.345803 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c7113-fc8c-4626-954b-c2d0c75830d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:55:26.347051 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c7113-fc8c-4626-954b-c2d0c75830d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:55:26.347051 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c7113-fc8c-4626-954b-c2d0c75830d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:55:26.347051 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c7113-fc8c-4626-954b-c2d0c75830d9 admin admin] PATH_INFO: `/v3/users/d935a4a30e4d44e0a392596452a5c1b6` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:55:26.348144 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c7113-fc8c-4626-954b-c2d0c75830d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d935a4a30e4d44e0a392596452a5c1b6)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:55:26.350055 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c7113-fc8c-4626-954b-c2d0c75830d9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:55:26.505288 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ad0c7113-fc8c-4626-954b-c2d0c75830d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd935a4a30e4d44e0a392596452a5c1b6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:55:26.565352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ad0c7113-fc8c-4626-954b-c2d0c75830d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd935a4a30e4d44e0a392596452a5c1b6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:55:26.565955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ad0c7113-fc8c-4626-954b-c2d0c75830d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd935a4a30e4d44e0a392596452a5c1b6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:55:26.566569 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ad0c7113-fc8c-4626-954b-c2d0c75830d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd935a4a30e4d44e0a392596452a5c1b6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:55:26.579086 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ad0c7113-fc8c-4626-954b-c2d0c75830d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd935a4a30e4d44e0a392596452a5c1b6'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:55:26.591286 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1312/2630] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:55:26 2022] DELETE /identity/v3/users/d935a4a30e4d44e0a392596452a5c1b6 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:55:26.825064 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936350cc-5062-4d55-9651-c5a0d05cb3f6 service placement] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:55:26.836239 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936350cc-5062-4d55-9651-c5a0d05cb3f6 service placement] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:55:26.893737 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6abee37-4e1e-4d09-a15f-0ce36bb99bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:55:26.894395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abee37-4e1e-4d09-a15f-0ce36bb99bc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:55:26.894678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abee37-4e1e-4d09-a15f-0ce36bb99bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:55:26.894982 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6abee37-4e1e-4d09-a15f-0ce36bb99bc9 admin admin] PATH_INFO: `/v3/projects/2c5938d62e2d40969d096b66e14edd75` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:55:26.905804 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6abee37-4e1e-4d09-a15f-0ce36bb99bc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c5938d62e2d40969d096b66e14edd75)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:55:26.907750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6abee37-4e1e-4d09-a15f-0ce36bb99bc9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:55:27.017678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e6abee37-4e1e-4d09-a15f-0ce36bb99bc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c5938d62e2d40969d096b66e14edd75', 'request_id': 'req-e6abee37-4e1e-4d09-a15f-0ce36bb99bc9'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:55:27.085667 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e6abee37-4e1e-4d09-a15f-0ce36bb99bc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c5938d62e2d40969d096b66e14edd75', 'request_id': 'req-e6abee37-4e1e-4d09-a15f-0ce36bb99bc9'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:55:27.091575 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e6abee37-4e1e-4d09-a15f-0ce36bb99bc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c5938d62e2d40969d096b66e14edd75', 'request_id': 'req-e6abee37-4e1e-4d09-a15f-0ce36bb99bc9'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:55:27.103718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e6abee37-4e1e-4d09-a15f-0ce36bb99bc9 admin admin] The token cache is being invalidate because project 2c5938d62e2d40969d096b66e14edd75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:55:27.107415 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e6abee37-4e1e-4d09-a15f-0ce36bb99bc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:55:27.107415 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e6abee37-4e1e-4d09-a15f-0ce36bb99bc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:55:27.107415 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1319/2631] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:55:26 2022] DELETE /identity/v3/projects/2c5938d62e2d40969d096b66e14edd75 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:55:27.110167 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c7113-fc8c-4626-954b-c2d0c75830d9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:55:27.111631 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ad0c7113-fc8c-4626-954b-c2d0c75830d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:55:27.124525 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c7113-fc8c-4626-954b-c2d0c75830d9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:55:27.132278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:55:27.180329 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339cfcba-7c16-4068-9f06-844e42580cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:55:27.180945 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339cfcba-7c16-4068-9f06-844e42580cf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:55:27.181223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339cfcba-7c16-4068-9f06-844e42580cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:55:27.181495 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339cfcba-7c16-4068-9f06-844e42580cf4 admin admin] PATH_INFO: `/v3/users/2ab56d1b8a804202bcea64a1d0105f99` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:55:27.194816 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339cfcba-7c16-4068-9f06-844e42580cf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ab56d1b8a804202bcea64a1d0105f99)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:55:27.196713 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339cfcba-7c16-4068-9f06-844e42580cf4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:55:27.382727 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-339cfcba-7c16-4068-9f06-844e42580cf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ab56d1b8a804202bcea64a1d0105f99'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:55:27.469022 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-339cfcba-7c16-4068-9f06-844e42580cf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ab56d1b8a804202bcea64a1d0105f99'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:55:27.469968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-339cfcba-7c16-4068-9f06-844e42580cf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ab56d1b8a804202bcea64a1d0105f99'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:55:27.470569 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-339cfcba-7c16-4068-9f06-844e42580cf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ab56d1b8a804202bcea64a1d0105f99'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:55:27.484813 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-339cfcba-7c16-4068-9f06-844e42580cf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ab56d1b8a804202bcea64a1d0105f99'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:55:27.507758 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1313/2632] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 09:55:27 2022] DELETE /identity/v3/users/2ab56d1b8a804202bcea64a1d0105f99 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 09:55:27.745289 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6abee37-4e1e-4d09-a15f-0ce36bb99bc9 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:55:27.747578 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e6abee37-4e1e-4d09-a15f-0ce36bb99bc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:55:27.760329 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6abee37-4e1e-4d09-a15f-0ce36bb99bc9 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:55:27.805851 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:55:27.822200 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b2e52-5acb-4a89-802c-3bea900b5b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:55:27.822890 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b2e52-5acb-4a89-802c-3bea900b5b46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:55:27.823173 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b2e52-5acb-4a89-802c-3bea900b5b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:55:27.823447 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b2e52-5acb-4a89-802c-3bea900b5b46 admin admin] PATH_INFO: `/v3/projects/bd528e84c5b24fe19e0f7fa9365c943a` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:55:27.833681 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b2e52-5acb-4a89-802c-3bea900b5b46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd528e84c5b24fe19e0f7fa9365c943a)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:55:27.835611 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b2e52-5acb-4a89-802c-3bea900b5b46 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:55:27.964591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ff3b2e52-5acb-4a89-802c-3bea900b5b46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd528e84c5b24fe19e0f7fa9365c943a', 'request_id': 'req-ff3b2e52-5acb-4a89-802c-3bea900b5b46'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:55:28.074207 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ff3b2e52-5acb-4a89-802c-3bea900b5b46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd528e84c5b24fe19e0f7fa9365c943a', 'request_id': 'req-ff3b2e52-5acb-4a89-802c-3bea900b5b46'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:55:28.080037 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ff3b2e52-5acb-4a89-802c-3bea900b5b46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd528e84c5b24fe19e0f7fa9365c943a', 'request_id': 'req-ff3b2e52-5acb-4a89-802c-3bea900b5b46'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:55:28.091553 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ff3b2e52-5acb-4a89-802c-3bea900b5b46 admin admin] The token cache is being invalidate because project bd528e84c5b24fe19e0f7fa9365c943a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 09:55:28.098948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ff3b2e52-5acb-4a89-802c-3bea900b5b46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:55:28.098948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ff3b2e52-5acb-4a89-802c-3bea900b5b46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 09:55:28.098948 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1320/2633] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 09:55:27 2022] DELETE /identity/v3/projects/bd528e84c5b24fe19e0f7fa9365c943a => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 09:55:37.495224 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:55:38.088525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=36,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:56:57.187231 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339cfcba-7c16-4068-9f06-844e42580cf4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:56:57.188619 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-339cfcba-7c16-4068-9f06-844e42580cf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:56:57.257953 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339cfcba-7c16-4068-9f06-844e42580cf4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:56:57.316987 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c4db9-adbf-4293-8fda-d07c207bd813 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5177fa4d168c4c219e09c7178bbbc790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:56:57.317594 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c4db9-adbf-4293-8fda-d07c207bd813 service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:56:57.317872 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c4db9-adbf-4293-8fda-d07c207bd813 service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:56:57.318163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c4db9-adbf-4293-8fda-d07c207bd813 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:56:57.319881 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-394c4db9-adbf-4293-8fda-d07c207bd813 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:56:57.330664 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c4db9-adbf-4293-8fda-d07c207bd813 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:56:57.332253 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c4db9-adbf-4293-8fda-d07c207bd813 service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:56:57.341761 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1314/2634] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:56:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:57:07.338010 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=58 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:57:18.524275 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b2e52-5acb-4a89-802c-3bea900b5b46 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:57:18.525710 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ff3b2e52-5acb-4a89-802c-3bea900b5b46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:57:18.622541 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b2e52-5acb-4a89-802c-3bea900b5b46 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:57:18.628746 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d173f7a-a091-41d6-95cc-57552cd33f62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:57:18.629561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d173f7a-a091-41d6-95cc-57552cd33f62 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:57:18.630512 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d173f7a-a091-41d6-95cc-57552cd33f62 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:57:18.631767 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d173f7a-a091-41d6-95cc-57552cd33f62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:57:18.633705 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7d173f7a-a091-41d6-95cc-57552cd33f62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:57:18.651324 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d173f7a-a091-41d6-95cc-57552cd33f62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:57:18.652212 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d173f7a-a091-41d6-95cc-57552cd33f62 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:57:18.665612 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1321/2635] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:57:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:57:28.659575 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=34 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:58:45.917982 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c4db9-adbf-4293-8fda-d07c207bd813 service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:58:45.919375 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-394c4db9-adbf-4293-8fda-d07c207bd813 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:58:45.987769 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c4db9-adbf-4293-8fda-d07c207bd813 service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:58:45.991238 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013a960-f603-4972-9ad7-273ccdbce183 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:58:45.991836 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013a960-f603-4972-9ad7-273ccdbce183 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:58:45.992111 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013a960-f603-4972-9ad7-273ccdbce183 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:58:45.992379 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013a960-f603-4972-9ad7-273ccdbce183 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:58:45.994155 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1013a960-f603-4972-9ad7-273ccdbce183 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:58:46.066908 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013a960-f603-4972-9ad7-273ccdbce183 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:58:46.068310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013a960-f603-4972-9ad7-273ccdbce183 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:58:46.077171 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1315/2636] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:58:45 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:58:46.569316 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d173f7a-a091-41d6-95cc-57552cd33f62 service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:58:46.582127 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d173f7a-a091-41d6-95cc-57552cd33f62 service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:58:46.587710 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e1a18-0eb0-4d1f-886e-6a7303c0a0a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:58:46.588309 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e1a18-0eb0-4d1f-886e-6a7303c0a0a1 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:58:46.588587 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e1a18-0eb0-4d1f-886e-6a7303c0a0a1 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:58:46.588867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e1a18-0eb0-4d1f-886e-6a7303c0a0a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:58:46.590608 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4b9e1a18-0eb0-4d1f-886e-6a7303c0a0a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:58:46.670842 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e1a18-0eb0-4d1f-886e-6a7303c0a0a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:58:46.672461 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9e1a18-0eb0-4d1f-886e-6a7303c0a0a1 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:58:46.691565 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1322/2637] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:58:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:58:46.694051 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013a960-f603-4972-9ad7-273ccdbce183 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:58:46.708312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013a960-f603-4972-9ad7-273ccdbce183 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:58:46.713905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e69e3-16d8-4500-af05-feb58f2dd004 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:58:46.714889 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e69e3-16d8-4500-af05-feb58f2dd004 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:58:46.715361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e69e3-16d8-4500-af05-feb58f2dd004 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:58:46.715846 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e69e3-16d8-4500-af05-feb58f2dd004 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:58:46.729375 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e69e3-16d8-4500-af05-feb58f2dd004 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:58:46.731991 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e69e3-16d8-4500-af05-feb58f2dd004 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:58:46.748006 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1316/2638] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:58:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:58:56.688570 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=41 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:58:56.773231 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=64 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:59:00.756109 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e1a18-0eb0-4d1f-886e-6a7303c0a0a1 service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:59:00.757847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4b9e1a18-0eb0-4d1f-886e-6a7303c0a0a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:59:00.827009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e1a18-0eb0-4d1f-886e-6a7303c0a0a1 service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:59:00.830658 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53638716-c1f6-4a54-8659-e72735dc0d96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:59:00.831274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53638716-c1f6-4a54-8659-e72735dc0d96 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:59:00.831553 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53638716-c1f6-4a54-8659-e72735dc0d96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:59:00.831822 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53638716-c1f6-4a54-8659-e72735dc0d96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:59:00.834193 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-53638716-c1f6-4a54-8659-e72735dc0d96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:59:00.844949 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53638716-c1f6-4a54-8659-e72735dc0d96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:59:00.846392 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53638716-c1f6-4a54-8659-e72735dc0d96 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:59:00.854941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1323/2639] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:59:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:59:10.850991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=31 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 09:59:44.334158 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e69e3-16d8-4500-af05-feb58f2dd004 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 09:59:44.335699 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-033e69e3-16d8-4500-af05-feb58f2dd004 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:59:44.362926 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e69e3-16d8-4500-af05-feb58f2dd004 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 09:59:44.368235 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75446445-114a-4d02-8327-e9b25d33d96e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 09:59:44.368828 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75446445-114a-4d02-8327-e9b25d33d96e service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 09:59:44.369139 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75446445-114a-4d02-8327-e9b25d33d96e service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 09:59:44.369414 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75446445-114a-4d02-8327-e9b25d33d96e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 09:59:44.371111 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-75446445-114a-4d02-8327-e9b25d33d96e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 09:59:44.427825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75446445-114a-4d02-8327-e9b25d33d96e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 09:59:44.429382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75446445-114a-4d02-8327-e9b25d33d96e service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 09:59:44.438827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1317/2640] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 09:59:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 09:59:54.434100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=33 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:00:17.883442 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53638716-c1f6-4a54-8659-e72735dc0d96 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:00:17.892176 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53638716-c1f6-4a54-8659-e72735dc0d96 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:00:17.895680 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8684516-8bd7-4ecd-82ab-02ad93c400b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:00:17.896268 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8684516-8bd7-4ecd-82ab-02ad93c400b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:00:17.896552 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8684516-8bd7-4ecd-82ab-02ad93c400b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:00:17.896811 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8684516-8bd7-4ecd-82ab-02ad93c400b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:00:17.898489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b8684516-8bd7-4ecd-82ab-02ad93c400b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:00:17.911862 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8684516-8bd7-4ecd-82ab-02ad93c400b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:00:17.913267 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8684516-8bd7-4ecd-82ab-02ad93c400b2 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:00:17.924479 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1324/2641] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:00:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:00:20.986595 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75446445-114a-4d02-8327-e9b25d33d96e service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:00:20.988082 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-75446445-114a-4d02-8327-e9b25d33d96e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:00:21.000061 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75446445-114a-4d02-8327-e9b25d33d96e service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:00:21.003626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf613d00-24b3-4a76-b45a-176595ad77f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:00:21.004224 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf613d00-24b3-4a76-b45a-176595ad77f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:00:21.004509 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf613d00-24b3-4a76-b45a-176595ad77f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:00:21.004777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf613d00-24b3-4a76-b45a-176595ad77f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:00:21.006887 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bf613d00-24b3-4a76-b45a-176595ad77f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:00:21.020407 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf613d00-24b3-4a76-b45a-176595ad77f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:00:21.021809 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf613d00-24b3-4a76-b45a-176595ad77f4 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:00:21.032501 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1318/2642] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:00:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:00:27.919775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:00:31.027735 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:02:00.159609 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8684516-8bd7-4ecd-82ab-02ad93c400b2 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:02:00.161023 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b8684516-8bd7-4ecd-82ab-02ad93c400b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:02:00.213103 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8684516-8bd7-4ecd-82ab-02ad93c400b2 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:02:00.218112 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8383f1-f0ef-4a15-9763-e4f729cfa423 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5177fa4d168c4c219e09c7178bbbc790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:02:00.218729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8383f1-f0ef-4a15-9763-e4f729cfa423 service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:02:00.219021 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8383f1-f0ef-4a15-9763-e4f729cfa423 service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:02:00.219295 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8383f1-f0ef-4a15-9763-e4f729cfa423 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:02:00.228249 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8383f1-f0ef-4a15-9763-e4f729cfa423 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:02:00.229817 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8383f1-f0ef-4a15-9763-e4f729cfa423 service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:02:00.239862 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1325/2643] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:02:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:02:10.235531 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=22 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:03:19.863602 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374fd84e-e987-4981-a95d-7c84796ddc37 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:03:19.864134 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374fd84e-e987-4981-a95d-7c84796ddc37 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:03:19.864134 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374fd84e-e987-4981-a95d-7c84796ddc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:03:19.908406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-374fd84e-e987-4981-a95d-7c84796ddc37 None None] MFA Rules not processed for user `f65429de28ae41578c51a6b9099fea5c`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:03:19.909601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-374fd84e-e987-4981-a95d-7c84796ddc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:03:19.917038 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1319/2644] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:03:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:03:20.216718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8383f1-f0ef-4a15-9763-e4f729cfa423 service placement] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:03:20.226034 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8383f1-f0ef-4a15-9763-e4f729cfa423 service placement] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:03:20.229595 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570cce1-abc7-434a-8834-2ebc3f3e2f86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:03:20.230199 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570cce1-abc7-434a-8834-2ebc3f3e2f86 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:03:20.230464 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570cce1-abc7-434a-8834-2ebc3f3e2f86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:03:20.230769 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570cce1-abc7-434a-8834-2ebc3f3e2f86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:03:20.232508 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b570cce1-abc7-434a-8834-2ebc3f3e2f86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:03:20.241720 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b570cce1-abc7-434a-8834-2ebc3f3e2f86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:03:20.243170 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b570cce1-abc7-434a-8834-2ebc3f3e2f86 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:03:20.251227 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1326/2645] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:03:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:03:20.647544 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374fd84e-e987-4981-a95d-7c84796ddc37 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:03:20.660969 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374fd84e-e987-4981-a95d-7c84796ddc37 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:03:20.666059 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d923a7-1ef6-4a8a-865e-cf2677fe11bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5177fa4d168c4c219e09c7178bbbc790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:03:20.666832 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d923a7-1ef6-4a8a-865e-cf2677fe11bb service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:03:20.667211 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d923a7-1ef6-4a8a-865e-cf2677fe11bb service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:03:20.667581 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d923a7-1ef6-4a8a-865e-cf2677fe11bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:03:20.669402 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f8d923a7-1ef6-4a8a-865e-cf2677fe11bb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:03:20.682095 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d923a7-1ef6-4a8a-865e-cf2677fe11bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:03:20.683757 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d923a7-1ef6-4a8a-865e-cf2677fe11bb service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:03:20.695313 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1320/2646] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:03:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:03:24.429729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570cce1-abc7-434a-8834-2ebc3f3e2f86 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:03:24.445400 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570cce1-abc7-434a-8834-2ebc3f3e2f86 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:03:24.450936 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc21bd6-68e3-467d-aa4d-4f2497b66d0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:03:24.451539 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc21bd6-68e3-467d-aa4d-4f2497b66d0d service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:03:24.451820 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc21bd6-68e3-467d-aa4d-4f2497b66d0d service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:03:24.452090 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc21bd6-68e3-467d-aa4d-4f2497b66d0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:03:24.461483 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc21bd6-68e3-467d-aa4d-4f2497b66d0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:03:24.463104 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc21bd6-68e3-467d-aa4d-4f2497b66d0d service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:03:24.473841 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1327/2647] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:03:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:03:30.690418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=17 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:03:31.536696 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d923a7-1ef6-4a8a-865e-cf2677fe11bb service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:03:31.548904 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d923a7-1ef6-4a8a-865e-cf2677fe11bb service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:03:31.555111 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56750459-999e-4a48-b70e-39a424509717 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:03:31.555736 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56750459-999e-4a48-b70e-39a424509717 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:03:31.556015 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56750459-999e-4a48-b70e-39a424509717 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:03:31.556286 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56750459-999e-4a48-b70e-39a424509717 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:03:31.557980 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-56750459-999e-4a48-b70e-39a424509717 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:03:31.569847 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56750459-999e-4a48-b70e-39a424509717 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:03:31.571437 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56750459-999e-4a48-b70e-39a424509717 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:03:31.582658 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1321/2648] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:03:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:03:34.469264 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=12 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:03:36.168554 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc21bd6-68e3-467d-aa4d-4f2497b66d0d service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:03:36.181030 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc21bd6-68e3-467d-aa4d-4f2497b66d0d service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:03:36.186731 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5b8d0-120c-4c25-892f-c92560a77737 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:03:36.187348 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5b8d0-120c-4c25-892f-c92560a77737 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:03:36.187629 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5b8d0-120c-4c25-892f-c92560a77737 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:03:36.187931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5b8d0-120c-4c25-892f-c92560a77737 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:03:36.189643 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-41b5b8d0-120c-4c25-892f-c92560a77737 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:03:36.202427 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5b8d0-120c-4c25-892f-c92560a77737 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:03:36.204041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b5b8d0-120c-4c25-892f-c92560a77737 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:03:36.215254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1328/2649] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:03:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:03:41.577323 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:03:45.167161 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56750459-999e-4a48-b70e-39a424509717 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:03:45.178145 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56750459-999e-4a48-b70e-39a424509717 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:03:45.183789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d2d63-f86f-42dd-89fc-6e57bc1bd1cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:03:45.184414 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d2d63-f86f-42dd-89fc-6e57bc1bd1cc service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:03:45.184691 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d2d63-f86f-42dd-89fc-6e57bc1bd1cc service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:03:45.184962 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d2d63-f86f-42dd-89fc-6e57bc1bd1cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:03:45.193281 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d2d63-f86f-42dd-89fc-6e57bc1bd1cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:03:45.194850 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d2d63-f86f-42dd-89fc-6e57bc1bd1cc service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:03:45.204194 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1322/2650] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:03:45 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:03:46.210783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:03:55.199699 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:04:03.883735 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5b8d0-120c-4c25-892f-c92560a77737 service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:04:03.913142 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b5b8d0-120c-4c25-892f-c92560a77737 service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:04:03.916732 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac1754-5ea4-42ea-883c-981fd36cea88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:04:03.917329 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac1754-5ea4-42ea-883c-981fd36cea88 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:04:03.917610 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac1754-5ea4-42ea-883c-981fd36cea88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:04:03.917883 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac1754-5ea4-42ea-883c-981fd36cea88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:04:03.924927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac1754-5ea4-42ea-883c-981fd36cea88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:04:03.926341 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fac1754-5ea4-42ea-883c-981fd36cea88 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:04:03.934307 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1329/2651] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:04:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:04:13.930614 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=15 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:04:46.942321 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d2d63-f86f-42dd-89fc-6e57bc1bd1cc service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:04:46.973990 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d2d63-f86f-42dd-89fc-6e57bc1bd1cc service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:04:46.977516 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7c37f-eab8-4e5c-b850-78cd7e3933c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:04:46.978113 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7c37f-eab8-4e5c-b850-78cd7e3933c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:04:46.978386 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7c37f-eab8-4e5c-b850-78cd7e3933c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:04:46.978670 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7c37f-eab8-4e5c-b850-78cd7e3933c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:04:46.999828 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f7c37f-eab8-4e5c-b850-78cd7e3933c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:04:47.001235 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f7c37f-eab8-4e5c-b850-78cd7e3933c4 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:04:47.028471 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1323/2652] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:04:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:04:57.025919 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=28 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:05:19.270292 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac1754-5ea4-42ea-883c-981fd36cea88 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:05:19.270292 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0fac1754-5ea4-42ea-883c-981fd36cea88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:05:19.293693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac1754-5ea4-42ea-883c-981fd36cea88 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:05:19.297286 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c421db2-9246-4b19-95f5-0c4a52f3021a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:05:19.303873 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c421db2-9246-4b19-95f5-0c4a52f3021a service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:05:19.303873 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c421db2-9246-4b19-95f5-0c4a52f3021a service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:05:19.303873 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c421db2-9246-4b19-95f5-0c4a52f3021a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:05:19.306681 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c421db2-9246-4b19-95f5-0c4a52f3021a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:05:19.308116 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c421db2-9246-4b19-95f5-0c4a52f3021a service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:05:19.316699 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1330/2653] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:05:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:05:19.963519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7c37f-eab8-4e5c-b850-78cd7e3933c4 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:05:19.973058 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7c37f-eab8-4e5c-b850-78cd7e3933c4 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:05:19.976578 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e0c27-604b-45a1-a4d2-c5338ebced1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:05:19.977173 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e0c27-604b-45a1-a4d2-c5338ebced1b service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:05:19.977450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e0c27-604b-45a1-a4d2-c5338ebced1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:05:19.977716 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e0c27-604b-45a1-a4d2-c5338ebced1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:05:19.995321 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e0c27-604b-45a1-a4d2-c5338ebced1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:05:19.996997 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e0c27-604b-45a1-a4d2-c5338ebced1b service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:05:20.019943 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1324/2654] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:05:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:05:23.329559 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c421db2-9246-4b19-95f5-0c4a52f3021a service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:05:23.338938 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c421db2-9246-4b19-95f5-0c4a52f3021a service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:05:23.342492 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f9700-3ede-4979-8a40-04cdad784ebf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:05:23.343109 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f9700-3ede-4979-8a40-04cdad784ebf service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:05:23.343389 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f9700-3ede-4979-8a40-04cdad784ebf service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:05:23.343660 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f9700-3ede-4979-8a40-04cdad784ebf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:05:23.354083 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f9700-3ede-4979-8a40-04cdad784ebf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:05:23.355669 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f9700-3ede-4979-8a40-04cdad784ebf service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:05:23.366875 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1331/2655] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:05:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:05:30.016772 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=13 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:05:33.361550 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=16 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:07:04.830420 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e0c27-604b-45a1-a4d2-c5338ebced1b service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:07:04.836155 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b57e0c27-604b-45a1-a4d2-c5338ebced1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:07:04.899388 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e0c27-604b-45a1-a4d2-c5338ebced1b service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:07:04.904860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e983ca68-d802-431a-89f4-eebbbb9852da service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5177fa4d168c4c219e09c7178bbbc790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:07:04.905458 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983ca68-d802-431a-89f4-eebbbb9852da service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:07:04.905731 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983ca68-d802-431a-89f4-eebbbb9852da service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:07:04.905999 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983ca68-d802-431a-89f4-eebbbb9852da service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:07:04.908183 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e983ca68-d802-431a-89f4-eebbbb9852da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:07:04.920081 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e983ca68-d802-431a-89f4-eebbbb9852da service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:07:04.921629 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e983ca68-d802-431a-89f4-eebbbb9852da service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:07:04.931497 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1325/2656] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:07:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:07:14.927193 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=32 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:08:47.092184 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f9700-3ede-4979-8a40-04cdad784ebf service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:08:47.093740 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f65f9700-3ede-4979-8a40-04cdad784ebf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:08:47.162195 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f9700-3ede-4979-8a40-04cdad784ebf service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:08:47.165749 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468d02a-64fc-48d1-ab52-d11bed81b06d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:08:47.166346 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468d02a-64fc-48d1-ab52-d11bed81b06d service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:08:47.166627 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468d02a-64fc-48d1-ab52-d11bed81b06d service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:08:47.166929 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468d02a-64fc-48d1-ab52-d11bed81b06d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:08:47.168712 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5468d02a-64fc-48d1-ab52-d11bed81b06d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:08:47.246327 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468d02a-64fc-48d1-ab52-d11bed81b06d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:08:47.247782 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468d02a-64fc-48d1-ab52-d11bed81b06d service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:08:47.265939 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1332/2657] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:08:47 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:08:50.980361 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e983ca68-d802-431a-89f4-eebbbb9852da service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:08:51.051264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e983ca68-d802-431a-89f4-eebbbb9852da service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:08:51.057183 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285e394-7de2-4222-8f44-182447fec13c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:08:51.057782 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285e394-7de2-4222-8f44-182447fec13c service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:08:51.058121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285e394-7de2-4222-8f44-182447fec13c service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:08:51.058378 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285e394-7de2-4222-8f44-182447fec13c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:08:51.060244 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2285e394-7de2-4222-8f44-182447fec13c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:08:51.073864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2285e394-7de2-4222-8f44-182447fec13c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:08:51.075744 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2285e394-7de2-4222-8f44-182447fec13c service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:08:51.085895 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1326/2658] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:08:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:08:56.410078 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df036178-2be1-41f1-8119-3af6080ff210 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:08:56.411630 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df036178-2be1-41f1-8119-3af6080ff210 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:08:56.411630 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df036178-2be1-41f1-8119-3af6080ff210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:08:56.464783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-df036178-2be1-41f1-8119-3af6080ff210 None None] MFA Rules not processed for user `889d4c60870447bda2926106026a0537`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:08:56.465929 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-df036178-2be1-41f1-8119-3af6080ff210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:08:56.515081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:08:56.535616 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1333/2659] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:08:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:08:56.877342 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285e394-7de2-4222-8f44-182447fec13c service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:08:56.886838 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285e394-7de2-4222-8f44-182447fec13c service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:08:56.890422 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4932b-75d7-48e7-9147-cd34f780c3a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:08:56.891064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4932b-75d7-48e7-9147-cd34f780c3a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:08:56.891340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4932b-75d7-48e7-9147-cd34f780c3a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:08:56.891611 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4932b-75d7-48e7-9147-cd34f780c3a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:08:56.893306 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-89e4932b-75d7-48e7-9147-cd34f780c3a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:08:56.903337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e4932b-75d7-48e7-9147-cd34f780c3a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:08:56.904774 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e4932b-75d7-48e7-9147-cd34f780c3a7 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:08:56.912754 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1327/2660] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:08:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:08:57.397421 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df036178-2be1-41f1-8119-3af6080ff210 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:08:57.411525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df036178-2be1-41f1-8119-3af6080ff210 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:08:57.417989 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e959438-d45b-4e2f-badd-b97a13e54706 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5177fa4d168c4c219e09c7178bbbc790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:08:57.418660 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e959438-d45b-4e2f-badd-b97a13e54706 service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:08:57.418980 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e959438-d45b-4e2f-badd-b97a13e54706 service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:08:57.419250 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e959438-d45b-4e2f-badd-b97a13e54706 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:08:57.429218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e959438-d45b-4e2f-badd-b97a13e54706 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:08:57.431137 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e959438-d45b-4e2f-badd-b97a13e54706 service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:08:57.443767 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1334/2661] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:08:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:09:01.526305 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4932b-75d7-48e7-9147-cd34f780c3a7 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:09:01.540926 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4932b-75d7-48e7-9147-cd34f780c3a7 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:09:01.546733 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b1b27-8e2c-4d49-9f43-b8ec1b8e01f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:09:01.547382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b1b27-8e2c-4d49-9f43-b8ec1b8e01f6 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:01.547663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b1b27-8e2c-4d49-9f43-b8ec1b8e01f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:01.548271 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b1b27-8e2c-4d49-9f43-b8ec1b8e01f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:01.558376 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b1b27-8e2c-4d49-9f43-b8ec1b8e01f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:09:01.564745 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b1b27-8e2c-4d49-9f43-b8ec1b8e01f6 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:09:01.572173 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1328/2662] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:09:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:09:05.984805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e959438-d45b-4e2f-badd-b97a13e54706 service placement] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:09:05.986566 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3e959438-d45b-4e2f-badd-b97a13e54706 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:09:06.055745 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e959438-d45b-4e2f-badd-b97a13e54706 service placement] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:09:06.059945 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a750ee5-dc88-49e5-a889-0993f88a68bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:09:06.064350 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a750ee5-dc88-49e5-a889-0993f88a68bc service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:06.064350 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a750ee5-dc88-49e5-a889-0993f88a68bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:06.064350 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a750ee5-dc88-49e5-a889-0993f88a68bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:06.064350 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2a750ee5-dc88-49e5-a889-0993f88a68bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:09:06.073674 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a750ee5-dc88-49e5-a889-0993f88a68bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:09:06.075152 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a750ee5-dc88-49e5-a889-0993f88a68bc service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:09:06.083350 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1335/2663] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:09:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:09:06.339484 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b1b27-8e2c-4d49-9f43-b8ec1b8e01f6 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:09:06.350363 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b1b27-8e2c-4d49-9f43-b8ec1b8e01f6 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:09:06.356379 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05efa12f-8939-4443-9d4f-b908b0205dd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:09:06.356986 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05efa12f-8939-4443-9d4f-b908b0205dd1 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:06.357264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05efa12f-8939-4443-9d4f-b908b0205dd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:06.357535 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05efa12f-8939-4443-9d4f-b908b0205dd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:06.359344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-05efa12f-8939-4443-9d4f-b908b0205dd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:09:06.450764 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05efa12f-8939-4443-9d4f-b908b0205dd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:09:06.452188 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05efa12f-8939-4443-9d4f-b908b0205dd1 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:09:06.472033 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1329/2664] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:09:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:09:09.883057 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a750ee5-dc88-49e5-a889-0993f88a68bc service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:09:09.898770 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a750ee5-dc88-49e5-a889-0993f88a68bc service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:09:09.906532 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a08b9c-90a0-46a0-9ca6-cf4c872e3094 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:09:09.907379 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a08b9c-90a0-46a0-9ca6-cf4c872e3094 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:09.907867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a08b9c-90a0-46a0-9ca6-cf4c872e3094 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:09.908136 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a08b9c-90a0-46a0-9ca6-cf4c872e3094 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:09.918307 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a08b9c-90a0-46a0-9ca6-cf4c872e3094 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:09:09.922359 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a08b9c-90a0-46a0-9ca6-cf4c872e3094 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:09:09.933288 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1336/2665] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:09:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:09:10.285512 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05efa12f-8939-4443-9d4f-b908b0205dd1 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:09:10.300327 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05efa12f-8939-4443-9d4f-b908b0205dd1 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:09:10.308081 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459fcd1f-a02a-4cc6-96e3-4ad87874ff4e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5177fa4d168c4c219e09c7178bbbc790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:09:10.308081 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459fcd1f-a02a-4cc6-96e3-4ad87874ff4e service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:10.308081 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459fcd1f-a02a-4cc6-96e3-4ad87874ff4e service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:10.308081 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459fcd1f-a02a-4cc6-96e3-4ad87874ff4e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:10.311027 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-459fcd1f-a02a-4cc6-96e3-4ad87874ff4e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:09:10.323684 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459fcd1f-a02a-4cc6-96e3-4ad87874ff4e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:09:10.325254 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459fcd1f-a02a-4cc6-96e3-4ad87874ff4e service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:09:10.339055 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1330/2666] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:09:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:09:10.449715 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a08b9c-90a0-46a0-9ca6-cf4c872e3094 service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:09:10.451325 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a0a08b9c-90a0-46a0-9ca6-cf4c872e3094 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:09:10.467019 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a08b9c-90a0-46a0-9ca6-cf4c872e3094 service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:09:10.471681 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3081948-7d8b-42ca-a555-952eb6da688e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:09:10.480217 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3081948-7d8b-42ca-a555-952eb6da688e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:10.480217 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3081948-7d8b-42ca-a555-952eb6da688e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:10.480217 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3081948-7d8b-42ca-a555-952eb6da688e admin admin] PATH_INFO: `/v3/users/c79aaf1f72634eefb5e89c8005afe54b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:10.480217 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3081948-7d8b-42ca-a555-952eb6da688e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c79aaf1f72634eefb5e89c8005afe54b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:09:10.480217 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3081948-7d8b-42ca-a555-952eb6da688e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:09:10.701285 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d3081948-7d8b-42ca-a555-952eb6da688e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c79aaf1f72634eefb5e89c8005afe54b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:09:10.708109 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d3081948-7d8b-42ca-a555-952eb6da688e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c79aaf1f72634eefb5e89c8005afe54b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:09:10.771746 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d3081948-7d8b-42ca-a555-952eb6da688e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c79aaf1f72634eefb5e89c8005afe54b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:09:10.772392 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d3081948-7d8b-42ca-a555-952eb6da688e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c79aaf1f72634eefb5e89c8005afe54b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:09:10.785350 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d3081948-7d8b-42ca-a555-952eb6da688e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c79aaf1f72634eefb5e89c8005afe54b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:09:10.791235 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1337/2667] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 10:09:10 2022] DELETE /identity/v3/users/c79aaf1f72634eefb5e89c8005afe54b => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:09:11.027176 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459fcd1f-a02a-4cc6-96e3-4ad87874ff4e service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:09:11.038162 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459fcd1f-a02a-4cc6-96e3-4ad87874ff4e service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:09:11.046620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:09:11.096980 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510becc7-ba2e-4e24-b90e-5e5324e59ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:09:11.097619 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510becc7-ba2e-4e24-b90e-5e5324e59ab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:11.097892 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510becc7-ba2e-4e24-b90e-5e5324e59ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:11.098163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510becc7-ba2e-4e24-b90e-5e5324e59ab7 admin admin] PATH_INFO: `/v3/projects/c2945ae512e14bfbb7d02dedc4de7cd4` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:11.099312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510becc7-ba2e-4e24-b90e-5e5324e59ab7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2945ae512e14bfbb7d02dedc4de7cd4)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:09:11.101192 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510becc7-ba2e-4e24-b90e-5e5324e59ab7 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:09:11.281237 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-510becc7-ba2e-4e24-b90e-5e5324e59ab7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2945ae512e14bfbb7d02dedc4de7cd4', 'request_id': 'req-510becc7-ba2e-4e24-b90e-5e5324e59ab7'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:09:11.286543 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-510becc7-ba2e-4e24-b90e-5e5324e59ab7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2945ae512e14bfbb7d02dedc4de7cd4', 'request_id': 'req-510becc7-ba2e-4e24-b90e-5e5324e59ab7'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:09:11.292924 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-510becc7-ba2e-4e24-b90e-5e5324e59ab7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2945ae512e14bfbb7d02dedc4de7cd4', 'request_id': 'req-510becc7-ba2e-4e24-b90e-5e5324e59ab7'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:09:11.360711 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-510becc7-ba2e-4e24-b90e-5e5324e59ab7 admin admin] The token cache is being invalidate because project c2945ae512e14bfbb7d02dedc4de7cd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 10:09:11.360994 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-510becc7-ba2e-4e24-b90e-5e5324e59ab7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:09:11.361614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-510becc7-ba2e-4e24-b90e-5e5324e59ab7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:09:11.363150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1331/2668] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 10:09:11 2022] DELETE /identity/v3/projects/c2945ae512e14bfbb7d02dedc4de7cd4 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:09:11.369103 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79e72e-233b-42cc-9211-8f8a50805aa1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:11.369619 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79e72e-233b-42cc-9211-8f8a50805aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:11.369748 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79e72e-233b-42cc-9211-8f8a50805aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:11.398600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:09:11.427806 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-db79e72e-233b-42cc-9211-8f8a50805aa1 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:09:11.429029 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-db79e72e-233b-42cc-9211-8f8a50805aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:09:11.488526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1338/2669] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:09:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:09:11.549633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232de613-d207-49a4-9610-c5c27129a415 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:11.549843 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232de613-d207-49a4-9610-c5c27129a415 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:11.550079 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232de613-d207-49a4-9610-c5c27129a415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:11.569358 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-232de613-d207-49a4-9610-c5c27129a415 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:09:11.570549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-232de613-d207-49a4-9610-c5c27129a415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:09:11.578851 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1332/2670] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:09:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:09:11.583007 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79e72e-233b-42cc-9211-8f8a50805aa1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:09:11.593145 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79e72e-233b-42cc-9211-8f8a50805aa1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:09:11.597703 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f2be8-dab9-4c5f-9c74-5ccc3d20e8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:09:11.598294 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f2be8-dab9-4c5f-9c74-5ccc3d20e8a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:11.598572 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f2be8-dab9-4c5f-9c74-5ccc3d20e8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:11.598880 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f2be8-dab9-4c5f-9c74-5ccc3d20e8a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:11.599516 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f2be8-dab9-4c5f-9c74-5ccc3d20e8a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 10:09:11.599816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f2be8-dab9-4c5f-9c74-5ccc3d20e8a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:09:11.601382 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f2be8-dab9-4c5f-9c74-5ccc3d20e8a5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:09:11.612504 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1339/2671] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 10:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 10:09:11.616873 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232de613-d207-49a4-9610-c5c27129a415 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:09:11.625910 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232de613-d207-49a4-9610-c5c27129a415 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:09:11.630475 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb9144-0e9d-4d8c-96f1-fe627f8c7695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:09:11.631085 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb9144-0e9d-4d8c-96f1-fe627f8c7695 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:11.631362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb9144-0e9d-4d8c-96f1-fe627f8c7695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:11.631625 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb9144-0e9d-4d8c-96f1-fe627f8c7695 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:11.632245 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb9144-0e9d-4d8c-96f1-fe627f8c7695 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:09:11.634143 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb9144-0e9d-4d8c-96f1-fe627f8c7695 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:09:11.750320 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1333/2672] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 10:09:11 2022] POST /identity/v3/projects => generated 387 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 10:09:11.754439 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f2be8-dab9-4c5f-9c74-5ccc3d20e8a5 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:09:11.765404 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f2be8-dab9-4c5f-9c74-5ccc3d20e8a5 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:09:11.829450 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025881ea-0699-4e70-af2b-d66c942eb9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:09:11.830062 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025881ea-0699-4e70-af2b-d66c942eb9cd admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:11.830344 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025881ea-0699-4e70-af2b-d66c942eb9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:11.830618 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025881ea-0699-4e70-af2b-d66c942eb9cd admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:11.831290 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025881ea-0699-4e70-af2b-d66c942eb9cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:09:11.833189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025881ea-0699-4e70-af2b-d66c942eb9cd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:09:11.948707 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1340/2673] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 10:09:11 2022] POST /identity/v3/users => generated 294 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 10:09:11.954426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb9144-0e9d-4d8c-96f1-fe627f8c7695 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:09:11.959768 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb9144-0e9d-4d8c-96f1-fe627f8c7695 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:09:11.964775 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3cc351-77b8-42a5-b842-bc9247f997a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:09:11.965362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3cc351-77b8-42a5-b842-bc9247f997a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:11.965638 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3cc351-77b8-42a5-b842-bc9247f997a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:11.965903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3cc351-77b8-42a5-b842-bc9247f997a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:11.966535 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3cc351-77b8-42a5-b842-bc9247f997a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:09:11.968131 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3cc351-77b8-42a5-b842-bc9247f997a5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:09:11.977741 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1334/2674] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 10:09:11.982083 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025881ea-0699-4e70-af2b-d66c942eb9cd admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:09:11.992517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025881ea-0699-4e70-af2b-d66c942eb9cd admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:09:11.997165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f136bfe-f8ff-42f0-923e-d836b8f526dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:09:11.997755 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f136bfe-f8ff-42f0-923e-d836b8f526dc admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:11.998040 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f136bfe-f8ff-42f0-923e-d836b8f526dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:11.998319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f136bfe-f8ff-42f0-923e-d836b8f526dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:11.998993 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f136bfe-f8ff-42f0-923e-d836b8f526dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:09:12.000568 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f136bfe-f8ff-42f0-923e-d836b8f526dc admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:09:12.011034 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1341/2675] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:09:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:09:12.014930 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3cc351-77b8-42a5-b842-bc9247f997a5 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:09:12.024516 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3cc351-77b8-42a5-b842-bc9247f997a5 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:09:12.029549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b158b-c699-45e7-b52e-dccd6f377bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:09:12.030184 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b158b-c699-45e7-b52e-dccd6f377bce admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:12.030461 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b158b-c699-45e7-b52e-dccd6f377bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:12.030769 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b158b-c699-45e7-b52e-dccd6f377bce admin admin] PATH_INFO: `/v3/projects/78a4f092ae254358b4e54190c2c4a1a1/users/cf7d0cc498e94ba49f258c38a3c4c410/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:12.045376 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b158b-c699-45e7-b52e-dccd6f377bce admin admin] RBAC: Authorizing `identity:create_grant(project_id=78a4f092ae254358b4e54190c2c4a1a1, user_id=cf7d0cc498e94ba49f258c38a3c4c410, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:09:12.047993 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b158b-c699-45e7-b52e-dccd6f377bce admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:09:12.144985 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1335/2676] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 10:09:12 2022] PUT /identity/v3/projects/78a4f092ae254358b4e54190c2c4a1a1/users/cf7d0cc498e94ba49f258c38a3c4c410/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:09:12.149952 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb635f84-e1bb-4059-9850-60048d820a9b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:12.150234 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb635f84-e1bb-4059-9850-60048d820a9b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:12.150508 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb635f84-e1bb-4059-9850-60048d820a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:12.170092 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-bb635f84-e1bb-4059-9850-60048d820a9b None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:09:12.171316 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bb635f84-e1bb-4059-9850-60048d820a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:09:12.218975 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:09:12.228284 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1342/2677] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:09:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:09:12.245820 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b158b-c699-45e7-b52e-dccd6f377bce admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:09:12.247715 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-104b158b-c699-45e7-b52e-dccd6f377bce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:09:12.282494 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:09:12.309017 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b158b-c699-45e7-b52e-dccd6f377bce admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:09:12.312862 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691206b3-f93a-4f57-91e5-afa6451b56b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:09:12.313473 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691206b3-f93a-4f57-91e5-afa6451b56b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:12.313746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691206b3-f93a-4f57-91e5-afa6451b56b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:12.314014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691206b3-f93a-4f57-91e5-afa6451b56b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:12.324738 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691206b3-f93a-4f57-91e5-afa6451b56b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:09:12.326155 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691206b3-f93a-4f57-91e5-afa6451b56b3 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:09:12.336679 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1336/2678] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:09:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:09:17.461260 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9ee41-7653-4274-8fce-cffbe4a8491a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:17.461944 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9ee41-7653-4274-8fce-cffbe4a8491a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:17.461944 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9ee41-7653-4274-8fce-cffbe4a8491a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:17.493097 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-64d9ee41-7653-4274-8fce-cffbe4a8491a None None] MFA Rules not processed for user `cf7d0cc498e94ba49f258c38a3c4c410`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:09:17.494424 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-64d9ee41-7653-4274-8fce-cffbe4a8491a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:09:17.562808 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1343/2679] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:09:17 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:09:17.582453 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691206b3-f93a-4f57-91e5-afa6451b56b3 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:09:17.584181 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-691206b3-f93a-4f57-91e5-afa6451b56b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:09:17.661573 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691206b3-f93a-4f57-91e5-afa6451b56b3 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:09:17.667462 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a85fd-51fd-438f-9d8c-c7956d631127 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:09:17.668084 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a85fd-51fd-438f-9d8c-c7956d631127 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:17.668366 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a85fd-51fd-438f-9d8c-c7956d631127 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:17.668642 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a85fd-51fd-438f-9d8c-c7956d631127 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:17.677412 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a85fd-51fd-438f-9d8c-c7956d631127 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:09:17.679166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a85fd-51fd-438f-9d8c-c7956d631127 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:09:17.689192 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1337/2680] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:09:17 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:09:20.061651 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73b5dd-d40d-4809-959b-04ca03e2039a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:20.062505 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73b5dd-d40d-4809-959b-04ca03e2039a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:20.062827 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd73b5dd-d40d-4809-959b-04ca03e2039a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:20.082447 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-bd73b5dd-d40d-4809-959b-04ca03e2039a None None] MFA Rules not processed for user `cf7d0cc498e94ba49f258c38a3c4c410`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:09:20.083694 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bd73b5dd-d40d-4809-959b-04ca03e2039a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:09:20.091778 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1344/2681] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:09:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:09:20.456365 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a85fd-51fd-438f-9d8c-c7956d631127 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:09:20.462862 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-605a85fd-51fd-438f-9d8c-c7956d631127 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:09:20.472628 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a85fd-51fd-438f-9d8c-c7956d631127 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:09:20.487690 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5209b65-65f8-4dca-ba6c-c9511366c087 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:09:20.491308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5209b65-65f8-4dca-ba6c-c9511366c087 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:20.491308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5209b65-65f8-4dca-ba6c-c9511366c087 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:20.491308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5209b65-65f8-4dca-ba6c-c9511366c087 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:20.491308 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f5209b65-65f8-4dca-ba6c-c9511366c087 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:09:20.504067 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5209b65-65f8-4dca-ba6c-c9511366c087 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:09:20.505509 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5209b65-65f8-4dca-ba6c-c9511366c087 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:09:20.515623 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1338/2682] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:09:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:09:30.076685 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=36 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:09:30.510601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=53 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:09:46.674540 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73b5dd-d40d-4809-959b-04ca03e2039a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:09:46.684681 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd73b5dd-d40d-4809-959b-04ca03e2039a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:09:46.688842 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ba5bf-b292-465c-be19-d20fe31145ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:09:46.689474 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ba5bf-b292-465c-be19-d20fe31145ee service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:46.689757 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ba5bf-b292-465c-be19-d20fe31145ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:46.690029 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ba5bf-b292-465c-be19-d20fe31145ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:46.691778 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-697ba5bf-b292-465c-be19-d20fe31145ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:09:46.703769 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ba5bf-b292-465c-be19-d20fe31145ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:09:46.705250 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ba5bf-b292-465c-be19-d20fe31145ee service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:09:46.715357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1345/2683] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:09:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:09:51.409225 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5209b65-65f8-4dca-ba6c-c9511366c087 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:09:51.410625 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f5209b65-65f8-4dca-ba6c-c9511366c087 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:09:51.480822 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5209b65-65f8-4dca-ba6c-c9511366c087 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:09:51.486269 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513883f3-80ad-4931-860a-14e0e46ecfd6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5177fa4d168c4c219e09c7178bbbc790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:09:51.486916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513883f3-80ad-4931-860a-14e0e46ecfd6 service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:51.489764 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513883f3-80ad-4931-860a-14e0e46ecfd6 service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:51.489764 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513883f3-80ad-4931-860a-14e0e46ecfd6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:51.489764 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-513883f3-80ad-4931-860a-14e0e46ecfd6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:09:51.501548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513883f3-80ad-4931-860a-14e0e46ecfd6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:09:51.503132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513883f3-80ad-4931-860a-14e0e46ecfd6 service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:09:51.513257 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1339/2684] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:09:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:09:51.552965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ba5bf-b292-465c-be19-d20fe31145ee service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:09:51.552965 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-697ba5bf-b292-465c-be19-d20fe31145ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:09:51.566930 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ba5bf-b292-465c-be19-d20fe31145ee service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:09:51.572202 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ba00c-1862-4e56-8f71-2cab2b485f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:09:51.572833 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ba00c-1862-4e56-8f71-2cab2b485f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:51.573117 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ba00c-1862-4e56-8f71-2cab2b485f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:51.573458 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ba00c-1862-4e56-8f71-2cab2b485f3e admin admin] PATH_INFO: `/v3/users/f65429de28ae41578c51a6b9099fea5c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:51.574521 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ba00c-1862-4e56-8f71-2cab2b485f3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f65429de28ae41578c51a6b9099fea5c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:09:51.576474 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ba00c-1862-4e56-8f71-2cab2b485f3e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:09:51.820578 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-588ba00c-1862-4e56-8f71-2cab2b485f3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f65429de28ae41578c51a6b9099fea5c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:09:51.828109 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-588ba00c-1862-4e56-8f71-2cab2b485f3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f65429de28ae41578c51a6b9099fea5c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:09:51.892568 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-588ba00c-1862-4e56-8f71-2cab2b485f3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f65429de28ae41578c51a6b9099fea5c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:09:51.893193 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-588ba00c-1862-4e56-8f71-2cab2b485f3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f65429de28ae41578c51a6b9099fea5c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:09:51.906438 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-588ba00c-1862-4e56-8f71-2cab2b485f3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f65429de28ae41578c51a6b9099fea5c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:09:51.913918 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1346/2685] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 10:09:51 2022] DELETE /identity/v3/users/f65429de28ae41578c51a6b9099fea5c => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:09:52.143077 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513883f3-80ad-4931-860a-14e0e46ecfd6 service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:09:52.153775 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513883f3-80ad-4931-860a-14e0e46ecfd6 service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:09:52.225296 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168742b-51cf-4710-aca8-bbe1c7ea9974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:09:52.225828 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168742b-51cf-4710-aca8-bbe1c7ea9974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:52.226009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168742b-51cf-4710-aca8-bbe1c7ea9974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:52.226284 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168742b-51cf-4710-aca8-bbe1c7ea9974 admin admin] PATH_INFO: `/v3/projects/ea33fe1eaf1047f78312a8e9fa9152c4` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:52.227480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3168742b-51cf-4710-aca8-bbe1c7ea9974 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea33fe1eaf1047f78312a8e9fa9152c4)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:09:52.229364 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3168742b-51cf-4710-aca8-bbe1c7ea9974 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:09:52.342121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3168742b-51cf-4710-aca8-bbe1c7ea9974 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea33fe1eaf1047f78312a8e9fa9152c4', 'request_id': 'req-3168742b-51cf-4710-aca8-bbe1c7ea9974'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:09:52.348207 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3168742b-51cf-4710-aca8-bbe1c7ea9974 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea33fe1eaf1047f78312a8e9fa9152c4', 'request_id': 'req-3168742b-51cf-4710-aca8-bbe1c7ea9974'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:09:52.354509 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3168742b-51cf-4710-aca8-bbe1c7ea9974 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea33fe1eaf1047f78312a8e9fa9152c4', 'request_id': 'req-3168742b-51cf-4710-aca8-bbe1c7ea9974'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:09:52.427675 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3168742b-51cf-4710-aca8-bbe1c7ea9974 admin admin] The token cache is being invalidate because project ea33fe1eaf1047f78312a8e9fa9152c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 10:09:52.428312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3168742b-51cf-4710-aca8-bbe1c7ea9974 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:09:52.428585 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3168742b-51cf-4710-aca8-bbe1c7ea9974 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:09:52.430112 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1340/2686] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 10:09:52 2022] DELETE /identity/v3/projects/ea33fe1eaf1047f78312a8e9fa9152c4 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:09:52.435101 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614114d-0a6e-4ca0-9d71-1a9c3be889ee None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:52.435378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614114d-0a6e-4ca0-9d71-1a9c3be889ee None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:52.435654 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614114d-0a6e-4ca0-9d71-1a9c3be889ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:52.455254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2614114d-0a6e-4ca0-9d71-1a9c3be889ee None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:09:52.456602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2614114d-0a6e-4ca0-9d71-1a9c3be889ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:09:52.514061 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1347/2687] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:09:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:09:52.522952 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d6aab-4a06-430a-a0de-1dce5c374e40 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:52.523347 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d6aab-4a06-430a-a0de-1dce5c374e40 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:52.523627 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00d6aab-4a06-430a-a0de-1dce5c374e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:52.543561 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d00d6aab-4a06-430a-a0de-1dce5c374e40 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:09:52.544751 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d00d6aab-4a06-430a-a0de-1dce5c374e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:09:52.553262 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1341/2688] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:09:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:09:52.557401 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614114d-0a6e-4ca0-9d71-1a9c3be889ee None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:09:52.567244 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614114d-0a6e-4ca0-9d71-1a9c3be889ee None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:09:52.572284 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a86dd-5532-4481-9b7d-770511ef460c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:09:52.572445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a86dd-5532-4481-9b7d-770511ef460c admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:52.572628 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a86dd-5532-4481-9b7d-770511ef460c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:52.572898 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a86dd-5532-4481-9b7d-770511ef460c admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:52.573551 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a86dd-5532-4481-9b7d-770511ef460c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 10:09:52.573857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a86dd-5532-4481-9b7d-770511ef460c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:09:52.575487 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a86dd-5532-4481-9b7d-770511ef460c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:09:52.586954 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1348/2689] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 10:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 10:09:52.590961 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d6aab-4a06-430a-a0de-1dce5c374e40 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:09:52.600578 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00d6aab-4a06-430a-a0de-1dce5c374e40 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:09:52.605070 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ebfca-83da-4ea5-9cfc-7c270fa920f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:09:52.605659 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ebfca-83da-4ea5-9cfc-7c270fa920f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:52.605934 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ebfca-83da-4ea5-9cfc-7c270fa920f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:52.606202 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ebfca-83da-4ea5-9cfc-7c270fa920f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:52.606854 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ebfca-83da-4ea5-9cfc-7c270fa920f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:09:52.609391 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ebfca-83da-4ea5-9cfc-7c270fa920f8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:09:52.620576 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:09:52.693513 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1342/2690] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 10:09:52 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:09:52.697476 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a86dd-5532-4481-9b7d-770511ef460c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:09:52.708279 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a86dd-5532-4481-9b7d-770511ef460c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:09:52.751988 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:09:52.765237 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9474e53d-2fb0-42fb-a3ab-c0153a570340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:09:52.765838 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474e53d-2fb0-42fb-a3ab-c0153a570340 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:52.766122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474e53d-2fb0-42fb-a3ab-c0153a570340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:52.766397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474e53d-2fb0-42fb-a3ab-c0153a570340 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:52.767070 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9474e53d-2fb0-42fb-a3ab-c0153a570340 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:09:52.769041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9474e53d-2fb0-42fb-a3ab-c0153a570340 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:09:52.862421 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1349/2691] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 10:09:52 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:09:52.866762 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ebfca-83da-4ea5-9cfc-7c270fa920f8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:09:52.877138 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ebfca-83da-4ea5-9cfc-7c270fa920f8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:09:52.881790 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d568a-c7a6-4ba5-b46a-6f4909012f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:09:52.882392 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d568a-c7a6-4ba5-b46a-6f4909012f30 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:52.882672 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d568a-c7a6-4ba5-b46a-6f4909012f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:52.882979 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6d568a-c7a6-4ba5-b46a-6f4909012f30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:52.883648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d568a-c7a6-4ba5-b46a-6f4909012f30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:09:52.885217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6d568a-c7a6-4ba5-b46a-6f4909012f30 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:09:52.895769 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1343/2692] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:09:52.899731 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9474e53d-2fb0-42fb-a3ab-c0153a570340 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:09:52.910127 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9474e53d-2fb0-42fb-a3ab-c0153a570340 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:09:52.914661 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afb56f-488e-4c0e-b979-8143b0b61221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:09:52.915278 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afb56f-488e-4c0e-b979-8143b0b61221 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:52.915557 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afb56f-488e-4c0e-b979-8143b0b61221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:52.915827 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afb56f-488e-4c0e-b979-8143b0b61221 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:52.916468 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80afb56f-488e-4c0e-b979-8143b0b61221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:09:52.918029 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80afb56f-488e-4c0e-b979-8143b0b61221 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:09:52.928390 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1350/2693] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:09:52.932435 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d568a-c7a6-4ba5-b46a-6f4909012f30 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:09:52.942212 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6d568a-c7a6-4ba5-b46a-6f4909012f30 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:09:52.946786 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77c0e6-dbb9-4461-a76e-00b15af30179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:09:52.947392 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77c0e6-dbb9-4461-a76e-00b15af30179 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:52.947674 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77c0e6-dbb9-4461-a76e-00b15af30179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:52.947950 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77c0e6-dbb9-4461-a76e-00b15af30179 admin admin] PATH_INFO: `/v3/projects/903eb64927ad43f1a2f1f2addb1c1b16/users/65963d36fc4741c69534b3dde1e00857/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:52.962515 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77c0e6-dbb9-4461-a76e-00b15af30179 admin admin] RBAC: Authorizing `identity:create_grant(project_id=903eb64927ad43f1a2f1f2addb1c1b16, user_id=65963d36fc4741c69534b3dde1e00857, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:09:52.965161 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea77c0e6-dbb9-4461-a76e-00b15af30179 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:09:53.044573 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1344/2694] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 10:09:52 2022] PUT /identity/v3/projects/903eb64927ad43f1a2f1f2addb1c1b16/users/65963d36fc4741c69534b3dde1e00857/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:09:53.049749 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e68d83-2e19-4f73-892f-034d036eed54 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:53.050032 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e68d83-2e19-4f73-892f-034d036eed54 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:53.050308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e68d83-2e19-4f73-892f-034d036eed54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:53.070641 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-44e68d83-2e19-4f73-892f-034d036eed54 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:09:53.071883 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-44e68d83-2e19-4f73-892f-034d036eed54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:09:53.252312 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1351/2695] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:09:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:09:53.261881 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77c0e6-dbb9-4461-a76e-00b15af30179 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:09:53.263570 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ea77c0e6-dbb9-4461-a76e-00b15af30179 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:09:53.318572 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea77c0e6-dbb9-4461-a76e-00b15af30179 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:09:53.322155 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45810286-8509-4cff-b088-be72157ca97f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:09:53.322779 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45810286-8509-4cff-b088-be72157ca97f service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:53.323062 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45810286-8509-4cff-b088-be72157ca97f service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:53.323334 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45810286-8509-4cff-b088-be72157ca97f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:53.332730 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45810286-8509-4cff-b088-be72157ca97f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:09:53.334134 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45810286-8509-4cff-b088-be72157ca97f service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:09:53.343286 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1345/2696] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:09:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:09:58.988565 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc4075-f0a3-4292-8df0-d652db27d2f7 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:58.989031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc4075-f0a3-4292-8df0-d652db27d2f7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:58.989190 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc4075-f0a3-4292-8df0-d652db27d2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:59.018741 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1edc4075-f0a3-4292-8df0-d652db27d2f7 None None] MFA Rules not processed for user `65963d36fc4741c69534b3dde1e00857`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:09:59.019893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1edc4075-f0a3-4292-8df0-d652db27d2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:09:59.087121 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1352/2697] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:09:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:09:59.103745 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45810286-8509-4cff-b088-be72157ca97f service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:09:59.105172 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-45810286-8509-4cff-b088-be72157ca97f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:09:59.199617 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45810286-8509-4cff-b088-be72157ca97f service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:09:59.205254 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c7908-e158-43c1-a9d5-c888f854262a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:09:59.207507 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c7908-e158-43c1-a9d5-c888f854262a service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:09:59.207507 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c7908-e158-43c1-a9d5-c888f854262a service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:09:59.207507 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c7908-e158-43c1-a9d5-c888f854262a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:09:59.214579 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c7908-e158-43c1-a9d5-c888f854262a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:09:59.216194 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c7908-e158-43c1-a9d5-c888f854262a service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:09:59.225859 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1346/2698] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:09:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:10:01.695202 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520624e2-cb30-41cd-bb3d-e05007e9beb7 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:10:01.695741 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520624e2-cb30-41cd-bb3d-e05007e9beb7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:10:01.695741 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520624e2-cb30-41cd-bb3d-e05007e9beb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:10:01.715647 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-520624e2-cb30-41cd-bb3d-e05007e9beb7 None None] MFA Rules not processed for user `65963d36fc4741c69534b3dde1e00857`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:10:01.716929 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-520624e2-cb30-41cd-bb3d-e05007e9beb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:10:01.724615 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1353/2699] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:10:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:10:09.221500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=83,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:10:11.709689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=75,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:10:17.574107 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c7908-e158-43c1-a9d5-c888f854262a service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:10:17.575753 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6d0c7908-e158-43c1-a9d5-c888f854262a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:10:17.589336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c7908-e158-43c1-a9d5-c888f854262a service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:10:17.592980 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689fd43-399b-4987-a7b8-985652f6a65d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:10:17.593575 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689fd43-399b-4987-a7b8-985652f6a65d service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:10:17.593853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689fd43-399b-4987-a7b8-985652f6a65d service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:10:17.594120 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0689fd43-399b-4987-a7b8-985652f6a65d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:10:17.596893 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0689fd43-399b-4987-a7b8-985652f6a65d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:10:17.609950 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689fd43-399b-4987-a7b8-985652f6a65d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:10:17.611497 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0689fd43-399b-4987-a7b8-985652f6a65d service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:10:17.623628 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1347/2700] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:10:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:10:18.368583 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520624e2-cb30-41cd-bb3d-e05007e9beb7 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:10:18.370237 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-520624e2-cb30-41cd-bb3d-e05007e9beb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:10:18.385099 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520624e2-cb30-41cd-bb3d-e05007e9beb7 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:10:18.389583 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77935869-cd1e-4dc5-9de0-fd9995ba9389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:10:18.392254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77935869-cd1e-4dc5-9de0-fd9995ba9389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:10:18.392254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77935869-cd1e-4dc5-9de0-fd9995ba9389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:10:18.392254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77935869-cd1e-4dc5-9de0-fd9995ba9389 admin admin] PATH_INFO: `/v3/users/889d4c60870447bda2926106026a0537` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:10:18.392254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77935869-cd1e-4dc5-9de0-fd9995ba9389 admin admin] RBAC: Authorizing `identity:delete_user(user_id=889d4c60870447bda2926106026a0537)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:10:18.393722 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77935869-cd1e-4dc5-9de0-fd9995ba9389 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:10:18.608216 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-77935869-cd1e-4dc5-9de0-fd9995ba9389 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '889d4c60870447bda2926106026a0537'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:10:18.616802 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-77935869-cd1e-4dc5-9de0-fd9995ba9389 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '889d4c60870447bda2926106026a0537'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:10:18.749517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-77935869-cd1e-4dc5-9de0-fd9995ba9389 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '889d4c60870447bda2926106026a0537'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:10:18.750273 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-77935869-cd1e-4dc5-9de0-fd9995ba9389 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '889d4c60870447bda2926106026a0537'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:10:18.762808 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-77935869-cd1e-4dc5-9de0-fd9995ba9389 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '889d4c60870447bda2926106026a0537'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:10:18.769265 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1354/2701] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 10:10:18 2022] DELETE /identity/v3/users/889d4c60870447bda2926106026a0537 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:10:19.090267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689fd43-399b-4987-a7b8-985652f6a65d service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:10:19.108746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0689fd43-399b-4987-a7b8-985652f6a65d service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:10:19.177862 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71001dfe-370f-48a6-afce-02831323e3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:10:19.178737 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71001dfe-370f-48a6-afce-02831323e3d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:10:19.179021 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71001dfe-370f-48a6-afce-02831323e3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:10:19.185281 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71001dfe-370f-48a6-afce-02831323e3d9 admin admin] PATH_INFO: `/v3/projects/7232d6277f26496ab552afa4abe13ad5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:10:19.185281 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71001dfe-370f-48a6-afce-02831323e3d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7232d6277f26496ab552afa4abe13ad5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:10:19.185281 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71001dfe-370f-48a6-afce-02831323e3d9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:10:19.311887 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-71001dfe-370f-48a6-afce-02831323e3d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7232d6277f26496ab552afa4abe13ad5', 'request_id': 'req-71001dfe-370f-48a6-afce-02831323e3d9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:10:19.318969 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-71001dfe-370f-48a6-afce-02831323e3d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7232d6277f26496ab552afa4abe13ad5', 'request_id': 'req-71001dfe-370f-48a6-afce-02831323e3d9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:10:19.325467 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-71001dfe-370f-48a6-afce-02831323e3d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7232d6277f26496ab552afa4abe13ad5', 'request_id': 'req-71001dfe-370f-48a6-afce-02831323e3d9'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:10:19.415830 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-71001dfe-370f-48a6-afce-02831323e3d9 admin admin] The token cache is being invalidate because project 7232d6277f26496ab552afa4abe13ad5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 10:10:19.416125 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-71001dfe-370f-48a6-afce-02831323e3d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:10:19.416820 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-71001dfe-370f-48a6-afce-02831323e3d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:10:19.418617 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1348/2702] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 10:10:19 2022] DELETE /identity/v3/projects/7232d6277f26496ab552afa4abe13ad5 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:10:28.765225 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:10:29.412616 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:11:07.208575 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77935869-cd1e-4dc5-9de0-fd9995ba9389 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:11:07.210001 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-77935869-cd1e-4dc5-9de0-fd9995ba9389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:11:07.292322 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77935869-cd1e-4dc5-9de0-fd9995ba9389 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:11:07.340029 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea0497-df31-4884-8e16-bfa8b89ebbd5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b555bb94ce44ce918cbf2a012576a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:11:07.340640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea0497-df31-4884-8e16-bfa8b89ebbd5 service glance] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:11:07.340919 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea0497-df31-4884-8e16-bfa8b89ebbd5 service glance] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:11:07.341189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea0497-df31-4884-8e16-bfa8b89ebbd5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:11:07.342945 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7cea0497-df31-4884-8e16-bfa8b89ebbd5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:11:07.411794 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea0497-df31-4884-8e16-bfa8b89ebbd5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:11:07.413238 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea0497-df31-4884-8e16-bfa8b89ebbd5 service glance] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:11:07.424970 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1355/2703] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:11:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:11:07.650950 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71001dfe-370f-48a6-afce-02831323e3d9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:11:07.652335 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-71001dfe-370f-48a6-afce-02831323e3d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:11:07.709600 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71001dfe-370f-48a6-afce-02831323e3d9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:11:07.713614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f65b93-ae02-4404-9abe-a25feaa33c93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:11:07.714219 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f65b93-ae02-4404-9abe-a25feaa33c93 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:11:07.714497 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f65b93-ae02-4404-9abe-a25feaa33c93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:11:07.714801 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f65b93-ae02-4404-9abe-a25feaa33c93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:11:07.716529 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-02f65b93-ae02-4404-9abe-a25feaa33c93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:11:07.730244 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f65b93-ae02-4404-9abe-a25feaa33c93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:11:07.731719 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f65b93-ae02-4404-9abe-a25feaa33c93 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:11:07.742647 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1349/2704] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:11:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:11:17.420172 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=83 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:11:17.737970 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=26 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:11:25.375462 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd0c3e-5a9a-44de-af0e-c5a4604e2210 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:11:25.375973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd0c3e-5a9a-44de-af0e-c5a4604e2210 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:11:25.376099 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd0c3e-5a9a-44de-af0e-c5a4604e2210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:11:25.440131 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-53fd0c3e-5a9a-44de-af0e-c5a4604e2210 None None] MFA Rules not processed for user `5233689a53364258b86ad2fa012d8db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:11:25.441363 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-53fd0c3e-5a9a-44de-af0e-c5a4604e2210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:11:25.519533 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1356/2705] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Sep 2 10:11:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:11:25.671827 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f65b93-ae02-4404-9abe-a25feaa33c93 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:11:25.673280 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-02f65b93-ae02-4404-9abe-a25feaa33c93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:11:25.765832 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f65b93-ae02-4404-9abe-a25feaa33c93 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:11:25.770782 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c6a1d6-a164-4897-9c40-cb56e28871d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30fe233a7d4c4fbea6ede233bd9ff8c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:11:25.771349 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6a1d6-a164-4897-9c40-cb56e28871d7 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:11:25.771625 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6a1d6-a164-4897-9c40-cb56e28871d7 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:11:25.771920 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c6a1d6-a164-4897-9c40-cb56e28871d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:11:25.779863 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6a1d6-a164-4897-9c40-cb56e28871d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:11:25.781485 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c6a1d6-a164-4897-9c40-cb56e28871d7 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:11:25.789927 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1350/2706] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Sep 2 10:11:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 10:11:35.516852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=44 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:11:35.786151 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=41 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:11:41.369417 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e670b-c0ac-47bd-92b4-2976ae293df1 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:11:41.370065 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e670b-c0ac-47bd-92b4-2976ae293df1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:11:41.370065 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e670b-c0ac-47bd-92b4-2976ae293df1 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:11:41.372118 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1357/2707] 10.222.0.57 () {54 vars in 921 bytes} [Fri Sep 2 10:11:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 10:11:41.378139 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2bff19-ad09-460f-a853-53a542fc5ccf None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:11:41.379196 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2bff19-ad09-460f-a853-53a542fc5ccf None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:11:41.379548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2bff19-ad09-460f-a853-53a542fc5ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:11:41.419626 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-4c2bff19-ad09-460f-a853-53a542fc5ccf None None] MFA Rules not processed for user `1da355c8be2d496bb08feb0fe74e04b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:11:41.420783 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4c2bff19-ad09-460f-a853-53a542fc5ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:11:41.430309 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1351/2708] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Sep 2 10:11:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 10:11:41.441817 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e670b-c0ac-47bd-92b4-2976ae293df1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:11:41.443855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-da4e670b-c0ac-47bd-92b4-2976ae293df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:11:41.457907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e670b-c0ac-47bd-92b4-2976ae293df1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:11:41.461423 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd09710-e93c-4c22-ae10-67648c0073b1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31b555bb94ce44ce918cbf2a012576a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:11:41.465002 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd09710-e93c-4c22-ae10-67648c0073b1 service glance] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:11:41.465002 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd09710-e93c-4c22-ae10-67648c0073b1 service glance] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:11:41.465002 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd09710-e93c-4c22-ae10-67648c0073b1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:11:41.473530 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd09710-e93c-4c22-ae10-67648c0073b1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:11:41.475097 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd09710-e93c-4c22-ae10-67648c0073b1 service glance] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:11:41.487502 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1358/2709] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:11:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:11:41.692182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d389d-a9d8-4df5-bfda-a4695f0fbc42 None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:11:41.692396 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d389d-a9d8-4df5-bfda-a4695f0fbc42 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:11:41.692651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902d389d-a9d8-4df5-bfda-a4695f0fbc42 None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:11:41.693999 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1352/2710] 10.222.0.57 () {54 vars in 921 bytes} [Fri Sep 2 10:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 10:11:41.699419 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035e60e-7b6c-4f49-9485-114901babe3b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:11:41.699710 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035e60e-7b6c-4f49-9485-114901babe3b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:11:41.699990 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035e60e-7b6c-4f49-9485-114901babe3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:11:41.739367 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3035e60e-7b6c-4f49-9485-114901babe3b None None] MFA Rules not processed for user `dfea2ee0c9e84b48ac9bafe3173d937a`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:11:41.740566 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3035e60e-7b6c-4f49-9485-114901babe3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:11:41.747327 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1359/2711] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Sep 2 10:11:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 10:11:41.758889 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d389d-a9d8-4df5-bfda-a4695f0fbc42 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:11:41.770398 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902d389d-a9d8-4df5-bfda-a4695f0fbc42 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:11:41.773907 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a5cc9-3686-4ac3-bae0-d4a1651b26cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:11:41.774501 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a5cc9-3686-4ac3-bae0-d4a1651b26cd service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:11:41.775430 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a5cc9-3686-4ac3-bae0-d4a1651b26cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:11:41.775716 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a5cc9-3686-4ac3-bae0-d4a1651b26cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:11:41.782601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a5cc9-3686-4ac3-bae0-d4a1651b26cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:11:41.784080 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a5cc9-3686-4ac3-bae0-d4a1651b26cd service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:11:41.791804 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1353/2712] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:11:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:11:41.886811 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53bb2d-b2ad-45db-9a8f-86631d04fd45 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:11:41.887096 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53bb2d-b2ad-45db-9a8f-86631d04fd45 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:11:41.887363 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a53bb2d-b2ad-45db-9a8f-86631d04fd45 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:11:41.888730 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1360/2713] 10.222.0.57 () {54 vars in 921 bytes} [Fri Sep 2 10:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 10:11:41.894328 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bf760-46ca-4761-b3b9-293a25db23f4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:11:41.894721 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bf760-46ca-4761-b3b9-293a25db23f4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:11:41.895031 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bf760-46ca-4761-b3b9-293a25db23f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:11:41.914667 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b00bf760-46ca-4761-b3b9-293a25db23f4 None None] MFA Rules not processed for user `1da355c8be2d496bb08feb0fe74e04b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:11:41.915884 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b00bf760-46ca-4761-b3b9-293a25db23f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:11:41.925466 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1354/2714] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Sep 2 10:11:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:11:41.936752 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a53bb2d-b2ad-45db-9a8f-86631d04fd45 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:11:41.948739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a53bb2d-b2ad-45db-9a8f-86631d04fd45 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:11:41.951566 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6fe1eb-4a6c-4f31-98c6-5ecb1df8867d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:11:41.952152 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fe1eb-4a6c-4f31-98c6-5ecb1df8867d service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:11:41.952425 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fe1eb-4a6c-4f31-98c6-5ecb1df8867d service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:11:41.952696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6fe1eb-4a6c-4f31-98c6-5ecb1df8867d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:11:41.962475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6fe1eb-4a6c-4f31-98c6-5ecb1df8867d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:11:41.963908 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6fe1eb-4a6c-4f31-98c6-5ecb1df8867d service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:11:41.974078 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1361/2715] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:11:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:11:44.047659 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c1285-744c-4b6d-8a83-93594f65e39f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:11:44.048190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c1285-744c-4b6d-8a83-93594f65e39f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:11:44.048190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417c1285-744c-4b6d-8a83-93594f65e39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:11:44.067064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-417c1285-744c-4b6d-8a83-93594f65e39f None None] MFA Rules not processed for user `65963d36fc4741c69534b3dde1e00857`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:11:44.068264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-417c1285-744c-4b6d-8a83-93594f65e39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:11:44.123373 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1355/2716] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:11:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:11:45.942683 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba88714-f058-4485-965c-399d4748c447 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:11:45.943199 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba88714-f058-4485-965c-399d4748c447 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:11:45.943315 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba88714-f058-4485-965c-399d4748c447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:11:45.963413 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-eba88714-f058-4485-965c-399d4748c447 None None] MFA Rules not processed for user `5233689a53364258b86ad2fa012d8db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:11:45.964617 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-eba88714-f058-4485-965c-399d4748c447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:11:45.971920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1362/2717] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Sep 2 10:11:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:11:45.984395 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c1285-744c-4b6d-8a83-93594f65e39f None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:11:45.996335 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417c1285-744c-4b6d-8a83-93594f65e39f None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:11:46.001149 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eaa15f-c790-4c22-ad51-11c35b5bb9eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30fe233a7d4c4fbea6ede233bd9ff8c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:11:46.001746 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eaa15f-c790-4c22-ad51-11c35b5bb9eb service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:11:46.002022 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eaa15f-c790-4c22-ad51-11c35b5bb9eb service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:11:46.002294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eaa15f-c790-4c22-ad51-11c35b5bb9eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:11:46.011244 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eaa15f-c790-4c22-ad51-11c35b5bb9eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:11:46.012813 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eaa15f-c790-4c22-ad51-11c35b5bb9eb service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:11:46.021021 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1356/2718] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Sep 2 10:11:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 10:11:55.957602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=24 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:11:56.017750 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=49 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:12:07.118046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba88714-f058-4485-965c-399d4748c447 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:12:07.119477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-eba88714-f058-4485-965c-399d4748c447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:12:07.226113 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba88714-f058-4485-965c-399d4748c447 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:12:07.232494 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d768f-e50d-4606-b9d2-a392914440c1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5177fa4d168c4c219e09c7178bbbc790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:12:07.233179 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d768f-e50d-4606-b9d2-a392914440c1 service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:12:07.233475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d768f-e50d-4606-b9d2-a392914440c1 service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:12:07.233755 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d768f-e50d-4606-b9d2-a392914440c1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:12:07.236275 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-873d768f-e50d-4606-b9d2-a392914440c1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:12:07.249997 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d768f-e50d-4606-b9d2-a392914440c1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:12:07.251626 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d768f-e50d-4606-b9d2-a392914440c1 service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:12:07.262362 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1363/2719] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:12:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:12:17.257607 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=48 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:12:18.392008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b0267-df6c-433b-9a0a-a01b092134c5 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:12:18.392521 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b0267-df6c-433b-9a0a-a01b092134c5 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:12:18.392521 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b0267-df6c-433b-9a0a-a01b092134c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:12:18.411142 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8c5b0267-df6c-433b-9a0a-a01b092134c5 None None] MFA Rules not processed for user `5233689a53364258b86ad2fa012d8db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:12:18.412331 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8c5b0267-df6c-433b-9a0a-a01b092134c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:12:18.419485 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1357/2720] 10.222.0.57 () {58 vars in 967 bytes} [Fri Sep 2 10:12:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:12:18.431743 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d768f-e50d-4606-b9d2-a392914440c1 service placement] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:12:18.443135 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d768f-e50d-4606-b9d2-a392914440c1 service placement] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:12:18.448949 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3abea4-cbdf-4cba-a278-79ff3e499864 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30fe233a7d4c4fbea6ede233bd9ff8c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:12:18.449658 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3abea4-cbdf-4cba-a278-79ff3e499864 service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:12:18.450004 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3abea4-cbdf-4cba-a278-79ff3e499864 service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:12:18.450401 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3abea4-cbdf-4cba-a278-79ff3e499864 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:12:18.459022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3abea4-cbdf-4cba-a278-79ff3e499864 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:12:18.460578 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3abea4-cbdf-4cba-a278-79ff3e499864 service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:12:18.468781 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1364/2721] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Sep 2 10:12:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 10:12:28.406191 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:12:28.465414 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:13:26.820530 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51091c56-d186-4ff6-b43d-ec264d35ce49 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:13:26.824633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51091c56-d186-4ff6-b43d-ec264d35ce49 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:13:26.824633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51091c56-d186-4ff6-b43d-ec264d35ce49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:13:26.844311 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-51091c56-d186-4ff6-b43d-ec264d35ce49 None None] MFA Rules not processed for user `cf7d0cc498e94ba49f258c38a3c4c410`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:13:26.845536 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-51091c56-d186-4ff6-b43d-ec264d35ce49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:13:26.900720 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1358/2722] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:13:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 10:13:36.895870 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=25 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:13:50.073201 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f743a59-d2ad-4047-9e9f-e29f48697e70 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:13:50.077088 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f743a59-d2ad-4047-9e9f-e29f48697e70 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:13:50.077088 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f743a59-d2ad-4047-9e9f-e29f48697e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:13:50.097285 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3f743a59-d2ad-4047-9e9f-e29f48697e70 None None] MFA Rules not processed for user `65963d36fc4741c69534b3dde1e00857`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:13:50.098485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3f743a59-d2ad-4047-9e9f-e29f48697e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:13:50.105321 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1365/2723] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:13:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 10:14:00.091478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:14:03.497376 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51091c56-d186-4ff6-b43d-ec264d35ce49 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:14:03.498826 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-51091c56-d186-4ff6-b43d-ec264d35ce49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:14:03.514389 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51091c56-d186-4ff6-b43d-ec264d35ce49 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:14:03.520346 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b969c-edd3-4005-bbd6-84aa3727ecec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:14:03.520945 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b969c-edd3-4005-bbd6-84aa3727ecec service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:14:03.521222 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b969c-edd3-4005-bbd6-84aa3727ecec service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:14:03.521493 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b969c-edd3-4005-bbd6-84aa3727ecec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:14:03.523237 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-5c9b969c-edd3-4005-bbd6-84aa3727ecec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:14:03.535919 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9b969c-edd3-4005-bbd6-84aa3727ecec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:14:03.537477 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9b969c-edd3-4005-bbd6-84aa3727ecec service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:14:03.548244 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1359/2724] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:14:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:14:13.543425 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:14:59.071726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f743a59-d2ad-4047-9e9f-e29f48697e70 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:14:59.075311 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3f743a59-d2ad-4047-9e9f-e29f48697e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:14:59.113869 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f743a59-d2ad-4047-9e9f-e29f48697e70 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:14:59.120172 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af771a0-d7a8-4495-a8a3-9562f3837c6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:14:59.120843 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af771a0-d7a8-4495-a8a3-9562f3837c6f service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:14:59.121196 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af771a0-d7a8-4495-a8a3-9562f3837c6f service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:14:59.121511 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af771a0-d7a8-4495-a8a3-9562f3837c6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:14:59.123331 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2af771a0-d7a8-4495-a8a3-9562f3837c6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:14:59.133477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af771a0-d7a8-4495-a8a3-9562f3837c6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:14:59.135127 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af771a0-d7a8-4495-a8a3-9562f3837c6f service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:14:59.143820 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1366/2725] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:14:59 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:15:09.139793 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=17 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:15:12.891138 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b969c-edd3-4005-bbd6-84aa3727ecec service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:15:12.921033 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b969c-edd3-4005-bbd6-84aa3727ecec service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:15:12.924617 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0486a4-6d29-449a-8bc9-42e0db520d19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:15:12.925218 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0486a4-6d29-449a-8bc9-42e0db520d19 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:15:12.925493 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0486a4-6d29-449a-8bc9-42e0db520d19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:15:12.925762 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0486a4-6d29-449a-8bc9-42e0db520d19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:15:12.927521 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2e0486a4-6d29-449a-8bc9-42e0db520d19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:15:12.937010 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0486a4-6d29-449a-8bc9-42e0db520d19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:15:12.938412 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0486a4-6d29-449a-8bc9-42e0db520d19 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:15:12.946428 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1360/2726] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:15:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:15:17.928854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af771a0-d7a8-4495-a8a3-9562f3837c6f service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:15:17.939722 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af771a0-d7a8-4495-a8a3-9562f3837c6f service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:15:17.945246 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf256a7-447c-4d30-bc52-d71b6add718d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:15:17.945836 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf256a7-447c-4d30-bc52-d71b6add718d service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:15:17.946114 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf256a7-447c-4d30-bc52-d71b6add718d service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:15:17.946387 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf256a7-447c-4d30-bc52-d71b6add718d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:15:17.948245 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cbf256a7-447c-4d30-bc52-d71b6add718d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:15:17.960196 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf256a7-447c-4d30-bc52-d71b6add718d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:15:17.961770 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf256a7-447c-4d30-bc52-d71b6add718d service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:15:17.972749 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1367/2727] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:15:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:15:18.497446 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0486a4-6d29-449a-8bc9-42e0db520d19 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:15:18.511134 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0486a4-6d29-449a-8bc9-42e0db520d19 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:15:18.516573 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96963d24-df98-4336-9180-e9a8c57303ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:15:18.517297 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96963d24-df98-4336-9180-e9a8c57303ec service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:15:18.517584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96963d24-df98-4336-9180-e9a8c57303ec service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:15:18.517856 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96963d24-df98-4336-9180-e9a8c57303ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:15:18.519601 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-96963d24-df98-4336-9180-e9a8c57303ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:15:18.530128 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96963d24-df98-4336-9180-e9a8c57303ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:15:18.531721 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96963d24-df98-4336-9180-e9a8c57303ec service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:15:18.540934 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1361/2728] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:15:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:15:19.320367 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431d711-e7ed-4f9f-b3dc-483b1ba7a206 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:15:19.320825 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431d711-e7ed-4f9f-b3dc-483b1ba7a206 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:15:19.320921 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431d711-e7ed-4f9f-b3dc-483b1ba7a206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:15:19.349249 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e431d711-e7ed-4f9f-b3dc-483b1ba7a206 None None] MFA Rules not processed for user `5233689a53364258b86ad2fa012d8db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:15:19.350434 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e431d711-e7ed-4f9f-b3dc-483b1ba7a206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:15:19.359027 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1368/2729] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Sep 2 10:15:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:15:19.510371 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96963d24-df98-4336-9180-e9a8c57303ec service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:15:19.521913 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96963d24-df98-4336-9180-e9a8c57303ec service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:15:19.527599 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd968d-e9c6-4bc8-9cc0-e6fab040ea14 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30fe233a7d4c4fbea6ede233bd9ff8c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:15:19.528191 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd968d-e9c6-4bc8-9cc0-e6fab040ea14 service swift] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:15:19.528471 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd968d-e9c6-4bc8-9cc0-e6fab040ea14 service swift] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:15:19.528743 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd968d-e9c6-4bc8-9cc0-e6fab040ea14 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:15:19.536635 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd968d-e9c6-4bc8-9cc0-e6fab040ea14 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:15:19.538194 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd968d-e9c6-4bc8-9cc0-e6fab040ea14 service swift] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:15:19.546686 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1362/2730] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Sep 2 10:15:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 10:15:29.344469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=13 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:15:29.543572 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=27 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:15:36.721696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431d711-e7ed-4f9f-b3dc-483b1ba7a206 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:15:36.735195 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431d711-e7ed-4f9f-b3dc-483b1ba7a206 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:15:36.740172 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df788993-10c7-4b12-9662-a85744f40d48 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5177fa4d168c4c219e09c7178bbbc790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:15:36.743015 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df788993-10c7-4b12-9662-a85744f40d48 service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:15:36.743015 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df788993-10c7-4b12-9662-a85744f40d48 service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:15:36.743015 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df788993-10c7-4b12-9662-a85744f40d48 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:15:36.743275 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-df788993-10c7-4b12-9662-a85744f40d48 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:15:36.754627 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df788993-10c7-4b12-9662-a85744f40d48 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:15:36.756232 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df788993-10c7-4b12-9662-a85744f40d48 service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:15:36.766301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1369/2731] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:15:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:15:39.401596 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03965d5b-6cf2-437f-8c5c-b2ac68b33f1c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:15:39.408490 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03965d5b-6cf2-437f-8c5c-b2ac68b33f1c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:15:39.408490 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03965d5b-6cf2-437f-8c5c-b2ac68b33f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:15:39.421936 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-03965d5b-6cf2-437f-8c5c-b2ac68b33f1c None None] MFA Rules not processed for user `5233689a53364258b86ad2fa012d8db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:15:39.423238 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-03965d5b-6cf2-437f-8c5c-b2ac68b33f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:15:39.438229 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1363/2732] 10.222.0.57 () {58 vars in 1009 bytes} [Fri Sep 2 10:15:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:15:39.450102 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df788993-10c7-4b12-9662-a85744f40d48 service placement] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:15:39.461688 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df788993-10c7-4b12-9662-a85744f40d48 service placement] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:15:39.466988 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e66cd1a-a42f-4fcb-a137-66647543eae8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30fe233a7d4c4fbea6ede233bd9ff8c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:15:39.467585 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66cd1a-a42f-4fcb-a137-66647543eae8 service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:15:39.467865 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66cd1a-a42f-4fcb-a137-66647543eae8 service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:15:39.468135 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66cd1a-a42f-4fcb-a137-66647543eae8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:15:39.476277 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e66cd1a-a42f-4fcb-a137-66647543eae8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:15:39.478073 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e66cd1a-a42f-4fcb-a137-66647543eae8 service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:15:39.486438 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1370/2733] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Sep 2 10:15:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 02 10:15:41.841813 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c414d0-e979-4b38-87e0-920dcca2e2b4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:15:41.842346 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c414d0-e979-4b38-87e0-920dcca2e2b4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:15:41.842346 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c414d0-e979-4b38-87e0-920dcca2e2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:15:41.863705 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-07c414d0-e979-4b38-87e0-920dcca2e2b4 None None] MFA Rules not processed for user `65963d36fc4741c69534b3dde1e00857`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:15:41.865071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-07c414d0-e979-4b38-87e0-920dcca2e2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:15:41.873068 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1364/2734] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:15:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:15:42.214481 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e66cd1a-a42f-4fcb-a137-66647543eae8 service swift] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:15:42.226581 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e66cd1a-a42f-4fcb-a137-66647543eae8 service swift] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:15:42.230477 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c3dfe-d9b8-4747-bcbf-99daaba5d041 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:15:42.231129 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c3dfe-d9b8-4747-bcbf-99daaba5d041 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:15:42.231410 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c3dfe-d9b8-4747-bcbf-99daaba5d041 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:15:42.231681 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c3dfe-d9b8-4747-bcbf-99daaba5d041 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:15:42.233459 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d66c3dfe-d9b8-4747-bcbf-99daaba5d041 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:15:42.243761 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66c3dfe-d9b8-4747-bcbf-99daaba5d041 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:15:42.246020 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66c3dfe-d9b8-4747-bcbf-99daaba5d041 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:15:42.255215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1371/2735] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:15:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:15:51.857969 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=9 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:15:52.251128 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=18 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:16:06.421769 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377896c-8462-4d4b-9ed1-78b0d6ddc65d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:16:06.422209 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377896c-8462-4d4b-9ed1-78b0d6ddc65d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:16:06.422293 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377896c-8462-4d4b-9ed1-78b0d6ddc65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:16:06.441625 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2377896c-8462-4d4b-9ed1-78b0d6ddc65d None None] MFA Rules not processed for user `5233689a53364258b86ad2fa012d8db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:16:06.442873 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2377896c-8462-4d4b-9ed1-78b0d6ddc65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:16:06.450119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1365/2736] 10.222.0.57 () {58 vars in 967 bytes} [Fri Sep 2 10:16:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:16:06.462797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c3dfe-d9b8-4747-bcbf-99daaba5d041 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:16:06.475031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c3dfe-d9b8-4747-bcbf-99daaba5d041 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:16:06.480275 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c3299-dd11-499d-aea3-da10b2b98c09 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30fe233a7d4c4fbea6ede233bd9ff8c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:16:06.480866 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c3299-dd11-499d-aea3-da10b2b98c09 service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:16:06.481147 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c3299-dd11-499d-aea3-da10b2b98c09 service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:16:06.481419 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c3299-dd11-499d-aea3-da10b2b98c09 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:16:06.489830 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c3299-dd11-499d-aea3-da10b2b98c09 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:16:06.491460 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c3299-dd11-499d-aea3-da10b2b98c09 service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:16:06.499878 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1372/2737] 10.222.0.57 () {60 vars in 1384 bytes} [Fri Sep 2 10:16:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 02 10:16:06.574100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377896c-8462-4d4b-9ed1-78b0d6ddc65d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:16:06.575552 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2377896c-8462-4d4b-9ed1-78b0d6ddc65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:16:06.587861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377896c-8462-4d4b-9ed1-78b0d6ddc65d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:16:06.591435 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7131a2-82d3-4853-afc5-f12c3d47166b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:16:06.592032 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7131a2-82d3-4853-afc5-f12c3d47166b service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:16:06.592311 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7131a2-82d3-4853-afc5-f12c3d47166b service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:16:06.592582 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7131a2-82d3-4853-afc5-f12c3d47166b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:16:06.603764 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7131a2-82d3-4853-afc5-f12c3d47166b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:16:06.605197 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7131a2-82d3-4853-afc5-f12c3d47166b service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:16:06.626380 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1366/2738] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:16:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:16:07.276657 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c3299-dd11-499d-aea3-da10b2b98c09 service swift] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:16:07.286214 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c3299-dd11-499d-aea3-da10b2b98c09 service swift] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:16:07.290059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532be41-385f-49e8-af1b-952881e85dca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:16:07.290653 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532be41-385f-49e8-af1b-952881e85dca service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:16:07.290966 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532be41-385f-49e8-af1b-952881e85dca service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:16:07.291235 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532be41-385f-49e8-af1b-952881e85dca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:16:07.301560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1532be41-385f-49e8-af1b-952881e85dca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:16:07.302998 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1532be41-385f-49e8-af1b-952881e85dca service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:16:07.314232 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1373/2739] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:16:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:16:12.549366 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7131a2-82d3-4853-afc5-f12c3d47166b service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:16:12.560548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7131a2-82d3-4853-afc5-f12c3d47166b service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:16:12.566689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980eea0f-9598-4ea5-acb4-4699eb9672d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:16:12.567379 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980eea0f-9598-4ea5-acb4-4699eb9672d4 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:16:12.567832 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980eea0f-9598-4ea5-acb4-4699eb9672d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:16:12.568111 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980eea0f-9598-4ea5-acb4-4699eb9672d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:16:12.569799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-980eea0f-9598-4ea5-acb4-4699eb9672d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:16:12.583141 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980eea0f-9598-4ea5-acb4-4699eb9672d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:16:12.584720 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980eea0f-9598-4ea5-acb4-4699eb9672d4 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:16:12.595611 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1367/2740] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:16:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:16:17.309010 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=12 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:16:22.591019 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=19 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:17:07.433539 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532be41-385f-49e8-af1b-952881e85dca service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:07.434993 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1532be41-385f-49e8-af1b-952881e85dca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:07.449551 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532be41-385f-49e8-af1b-952881e85dca service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:07.455258 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61198e-0743-4f7c-abc9-85ef75e01fe7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5177fa4d168c4c219e09c7178bbbc790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:07.455859 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61198e-0743-4f7c-abc9-85ef75e01fe7 service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:07.456139 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61198e-0743-4f7c-abc9-85ef75e01fe7 service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:07.456408 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61198e-0743-4f7c-abc9-85ef75e01fe7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:07.465030 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd61198e-0743-4f7c-abc9-85ef75e01fe7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:07.466587 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd61198e-0743-4f7c-abc9-85ef75e01fe7 service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:07.476692 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1374/2741] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:17:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:17:12.607438 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980eea0f-9598-4ea5-acb4-4699eb9672d4 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:12.617267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980eea0f-9598-4ea5-acb4-4699eb9672d4 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:12.620784 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae0259-da17-4ab3-b18f-b08ad8eef305 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:12.621371 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae0259-da17-4ab3-b18f-b08ad8eef305 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:12.621646 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae0259-da17-4ab3-b18f-b08ad8eef305 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:12.621946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae0259-da17-4ab3-b18f-b08ad8eef305 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:12.623682 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-84ae0259-da17-4ab3-b18f-b08ad8eef305 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:12.689555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae0259-da17-4ab3-b18f-b08ad8eef305 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:12.690965 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae0259-da17-4ab3-b18f-b08ad8eef305 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:12.700479 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1368/2742] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:17:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:17:16.448683 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61198e-0743-4f7c-abc9-85ef75e01fe7 service placement] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:16.467230 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd61198e-0743-4f7c-abc9-85ef75e01fe7 service placement] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:16.473300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb01ba-de2e-43ab-adec-4d4b823d6acb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5177fa4d168c4c219e09c7178bbbc790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:16.475061 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb01ba-de2e-43ab-adec-4d4b823d6acb service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:16.475708 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb01ba-de2e-43ab-adec-4d4b823d6acb service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:16.476121 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb01ba-de2e-43ab-adec-4d4b823d6acb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:16.478888 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-dbdb01ba-de2e-43ab-adec-4d4b823d6acb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:16.492669 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb01ba-de2e-43ab-adec-4d4b823d6acb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:16.494236 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb01ba-de2e-43ab-adec-4d4b823d6acb service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:16.507358 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1375/2743] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:17:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:17:16.585061 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae0259-da17-4ab3-b18f-b08ad8eef305 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:16.586515 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-84ae0259-da17-4ab3-b18f-b08ad8eef305 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:16.601868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae0259-da17-4ab3-b18f-b08ad8eef305 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:16.607154 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4cf5fb-4652-47b0-9c1b-739755fa021a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:16.607922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cf5fb-4652-47b0-9c1b-739755fa021a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:16.608280 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cf5fb-4652-47b0-9c1b-739755fa021a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:16.608697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4cf5fb-4652-47b0-9c1b-739755fa021a admin admin] PATH_INFO: `/v3/users/cf7d0cc498e94ba49f258c38a3c4c410` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:16.610106 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4cf5fb-4652-47b0-9c1b-739755fa021a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf7d0cc498e94ba49f258c38a3c4c410)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:16.617346 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4cf5fb-4652-47b0-9c1b-739755fa021a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:16.791634 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3a4cf5fb-4652-47b0-9c1b-739755fa021a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf7d0cc498e94ba49f258c38a3c4c410'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:16.834418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3a4cf5fb-4652-47b0-9c1b-739755fa021a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf7d0cc498e94ba49f258c38a3c4c410'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:16.835130 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3a4cf5fb-4652-47b0-9c1b-739755fa021a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf7d0cc498e94ba49f258c38a3c4c410'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:16.835801 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3a4cf5fb-4652-47b0-9c1b-739755fa021a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf7d0cc498e94ba49f258c38a3c4c410'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:16.850266 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-3a4cf5fb-4652-47b0-9c1b-739755fa021a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf7d0cc498e94ba49f258c38a3c4c410'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:16.869521 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1369/2744] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 10:17:16 2022] DELETE /identity/v3/users/cf7d0cc498e94ba49f258c38a3c4c410 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:17:17.139348 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb01ba-de2e-43ab-adec-4d4b823d6acb service placement] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:17.162046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb01ba-de2e-43ab-adec-4d4b823d6acb service placement] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:17.218010 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a69f0-8c49-429c-85f4-38970d4736b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:17.218632 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a69f0-8c49-429c-85f4-38970d4736b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:17.218942 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a69f0-8c49-429c-85f4-38970d4736b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:17.219219 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a69f0-8c49-429c-85f4-38970d4736b1 admin admin] PATH_INFO: `/v3/projects/78a4f092ae254358b4e54190c2c4a1a1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:17.220927 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a69f0-8c49-429c-85f4-38970d4736b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78a4f092ae254358b4e54190c2c4a1a1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:17.222842 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a69f0-8c49-429c-85f4-38970d4736b1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:17.324692 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-aa6a69f0-8c49-429c-85f4-38970d4736b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78a4f092ae254358b4e54190c2c4a1a1', 'request_id': 'req-aa6a69f0-8c49-429c-85f4-38970d4736b1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:17.399089 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-aa6a69f0-8c49-429c-85f4-38970d4736b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78a4f092ae254358b4e54190c2c4a1a1', 'request_id': 'req-aa6a69f0-8c49-429c-85f4-38970d4736b1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:17.404978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-aa6a69f0-8c49-429c-85f4-38970d4736b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78a4f092ae254358b4e54190c2c4a1a1', 'request_id': 'req-aa6a69f0-8c49-429c-85f4-38970d4736b1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:17.417045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-aa6a69f0-8c49-429c-85f4-38970d4736b1 admin admin] The token cache is being invalidate because project 78a4f092ae254358b4e54190c2c4a1a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 10:17:17.417333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-aa6a69f0-8c49-429c-85f4-38970d4736b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:17.417999 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-aa6a69f0-8c49-429c-85f4-38970d4736b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:17.419517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1376/2745] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 10:17:17 2022] DELETE /identity/v3/projects/78a4f092ae254358b4e54190c2c4a1a1 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:17:17.424475 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6025b-3a6b-4534-844b-f219b1008d86 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:17.424784 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6025b-3a6b-4534-844b-f219b1008d86 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:17.425053 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b6025b-3a6b-4534-844b-f219b1008d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:17.444543 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c0b6025b-3a6b-4534-844b-f219b1008d86 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:17:17.445735 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c0b6025b-3a6b-4534-844b-f219b1008d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:17.519229 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1370/2746] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:17:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:17:17.527114 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba89eb-3603-4c38-aef1-12fbdb356686 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:17.527578 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba89eb-3603-4c38-aef1-12fbdb356686 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:17.527673 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba89eb-3603-4c38-aef1-12fbdb356686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:17.547069 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f2ba89eb-3603-4c38-aef1-12fbdb356686 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:17:17.548249 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f2ba89eb-3603-4c38-aef1-12fbdb356686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:17.555921 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1377/2747] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:17:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:17:17.560156 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6025b-3a6b-4534-844b-f219b1008d86 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:17.569766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b6025b-3a6b-4534-844b-f219b1008d86 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:17.574380 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2332f2-9bee-46d9-aaf8-621e6302b2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:17.575002 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2332f2-9bee-46d9-aaf8-621e6302b2d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:17.575289 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2332f2-9bee-46d9-aaf8-621e6302b2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:17.575567 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2332f2-9bee-46d9-aaf8-621e6302b2d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:17.576221 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2332f2-9bee-46d9-aaf8-621e6302b2d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 10:17:17.576548 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2332f2-9bee-46d9-aaf8-621e6302b2d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:17.578125 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2332f2-9bee-46d9-aaf8-621e6302b2d9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:17.580101 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:17:17.590449 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1371/2748] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 10:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 10:17:17.596460 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba89eb-3603-4c38-aef1-12fbdb356686 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:17.605736 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba89eb-3603-4c38-aef1-12fbdb356686 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:17.610263 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb39776-7527-46d0-bf89-fe0e78e138db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:17.610886 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb39776-7527-46d0-bf89-fe0e78e138db admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:17.611165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb39776-7527-46d0-bf89-fe0e78e138db admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:17.611436 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb39776-7527-46d0-bf89-fe0e78e138db admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:17.612063 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb39776-7527-46d0-bf89-fe0e78e138db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:17.613947 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb39776-7527-46d0-bf89-fe0e78e138db admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:17.736538 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1378/2749] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 10:17:17 2022] POST /identity/v3/projects => generated 387 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:17:17.742752 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2332f2-9bee-46d9-aaf8-621e6302b2d9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:17.754113 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2332f2-9bee-46d9-aaf8-621e6302b2d9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:17.809520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457f0a6-52d5-4db6-87cf-659be2960066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:17.810128 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457f0a6-52d5-4db6-87cf-659be2960066 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:17.810411 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457f0a6-52d5-4db6-87cf-659be2960066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:17.810690 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457f0a6-52d5-4db6-87cf-659be2960066 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:17.811351 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457f0a6-52d5-4db6-87cf-659be2960066 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:17.813805 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457f0a6-52d5-4db6-87cf-659be2960066 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:17.920136 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1372/2750] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 10:17:17 2022] POST /identity/v3/users => generated 294 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:17:17.924588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb39776-7527-46d0-bf89-fe0e78e138db admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:17.935005 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb39776-7527-46d0-bf89-fe0e78e138db admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:17.940174 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dacd70-346d-45cd-b4c9-9214991830f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:17.940764 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dacd70-346d-45cd-b4c9-9214991830f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:17.941186 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dacd70-346d-45cd-b4c9-9214991830f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:17.941472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dacd70-346d-45cd-b4c9-9214991830f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:17.942107 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dacd70-346d-45cd-b4c9-9214991830f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:17.943693 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dacd70-346d-45cd-b4c9-9214991830f1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:17.953400 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1379/2751] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:17:17.957951 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457f0a6-52d5-4db6-87cf-659be2960066 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:17.968101 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457f0a6-52d5-4db6-87cf-659be2960066 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:17.972908 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d09ff1-f07b-4cba-ab97-d598a681228b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:17.973496 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d09ff1-f07b-4cba-ab97-d598a681228b admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:17.973777 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d09ff1-f07b-4cba-ab97-d598a681228b admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:17.974048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d09ff1-f07b-4cba-ab97-d598a681228b admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:17.974974 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d09ff1-f07b-4cba-ab97-d598a681228b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:17.976531 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d09ff1-f07b-4cba-ab97-d598a681228b admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:17.988140 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1373/2752] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:17:17.992107 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dacd70-346d-45cd-b4c9-9214991830f1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:18.002473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dacd70-346d-45cd-b4c9-9214991830f1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:18.007049 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777f4f8-b87a-41c9-adc5-b16166192555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:18.007691 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777f4f8-b87a-41c9-adc5-b16166192555 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:18.007972 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777f4f8-b87a-41c9-adc5-b16166192555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:18.008273 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4777f4f8-b87a-41c9-adc5-b16166192555 admin admin] PATH_INFO: `/v3/projects/346b674017524e15b1f47266c683a341/users/aca032f7c5b940b094b86d026a522518/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:18.024170 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4777f4f8-b87a-41c9-adc5-b16166192555 admin admin] RBAC: Authorizing `identity:create_grant(project_id=346b674017524e15b1f47266c683a341, user_id=aca032f7c5b940b094b86d026a522518, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:18.026876 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4777f4f8-b87a-41c9-adc5-b16166192555 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:18.127842 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:17:18.132604 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1380/2753] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 10:17:17 2022] PUT /identity/v3/projects/346b674017524e15b1f47266c683a341/users/aca032f7c5b940b094b86d026a522518/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:17:18.145778 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03ee76-fc4b-4b0b-b638-78f1e0041b0b None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:18.146190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03ee76-fc4b-4b0b-b638-78f1e0041b0b None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:18.146637 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03ee76-fc4b-4b0b-b638-78f1e0041b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:18.182681 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-fb03ee76-fc4b-4b0b-b638-78f1e0041b0b None None] MFA Rules not processed for user `aca032f7c5b940b094b86d026a522518`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:17:18.183945 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fb03ee76-fc4b-4b0b-b638-78f1e0041b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:18.281821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1374/2754] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:17:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 10:17:18.286865 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777f4f8-b87a-41c9-adc5-b16166192555 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:18.347606 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4777f4f8-b87a-41c9-adc5-b16166192555 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:18.352504 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec0f97-0295-4750-9851-6634b8924ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:18.353097 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec0f97-0295-4750-9851-6634b8924ed9 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:18.353369 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec0f97-0295-4750-9851-6634b8924ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:18.353635 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec0f97-0295-4750-9851-6634b8924ed9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:18.354259 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ec0f97-0295-4750-9851-6634b8924ed9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:18.356156 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ec0f97-0295-4750-9851-6634b8924ed9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:18.468261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1381/2755] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 10:17:18 2022] POST /identity/v3/projects => generated 387 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 10:17:18.472311 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03ee76-fc4b-4b0b-b638-78f1e0041b0b None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:18.483527 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03ee76-fc4b-4b0b-b638-78f1e0041b0b None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:18.558274 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ad1a4-1c0d-48af-ad72-22b83ad52102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:18.558990 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ad1a4-1c0d-48af-ad72-22b83ad52102 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:18.559348 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ad1a4-1c0d-48af-ad72-22b83ad52102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:18.559696 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ad1a4-1c0d-48af-ad72-22b83ad52102 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:18.560482 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499ad1a4-1c0d-48af-ad72-22b83ad52102 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:18.562779 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499ad1a4-1c0d-48af-ad72-22b83ad52102 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:18.573175 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:17:18.679783 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1375/2756] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 10:17:18 2022] POST /identity/v3/users => generated 294 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:17:18.683936 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec0f97-0295-4750-9851-6634b8924ed9 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:18.694679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec0f97-0295-4750-9851-6634b8924ed9 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:18.699196 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a120f-17cc-488b-bdeb-e9b12b3adbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:18.699790 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a120f-17cc-488b-bdeb-e9b12b3adbd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:18.700064 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a120f-17cc-488b-bdeb-e9b12b3adbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:18.700331 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a120f-17cc-488b-bdeb-e9b12b3adbd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:18.701110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308a120f-17cc-488b-bdeb-e9b12b3adbd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:18.703538 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308a120f-17cc-488b-bdeb-e9b12b3adbd9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:18.714929 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1382/2757] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 10:17:18.720267 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ad1a4-1c0d-48af-ad72-22b83ad52102 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:18.730437 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499ad1a4-1c0d-48af-ad72-22b83ad52102 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:18.735006 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3db164-88ab-4bf9-b379-da586fea191c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:18.735591 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3db164-88ab-4bf9-b379-da586fea191c admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:18.735867 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3db164-88ab-4bf9-b379-da586fea191c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:18.736144 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3db164-88ab-4bf9-b379-da586fea191c admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:18.736771 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3db164-88ab-4bf9-b379-da586fea191c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:18.738331 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3db164-88ab-4bf9-b379-da586fea191c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:18.748050 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1376/2758] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:17:18.752664 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a120f-17cc-488b-bdeb-e9b12b3adbd9 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:18.764939 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a120f-17cc-488b-bdeb-e9b12b3adbd9 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:18.770359 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59944a2f-174a-492b-99e9-177d4983b749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:18.771039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59944a2f-174a-492b-99e9-177d4983b749 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:18.771315 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59944a2f-174a-492b-99e9-177d4983b749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:18.771583 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59944a2f-174a-492b-99e9-177d4983b749 admin admin] PATH_INFO: `/v3/projects/6170aac3eb2c4297b4e244f50c007963/users/d2c4d5b3e29a40bf8326ac902aef52f2/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:18.785721 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59944a2f-174a-492b-99e9-177d4983b749 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6170aac3eb2c4297b4e244f50c007963, user_id=d2c4d5b3e29a40bf8326ac902aef52f2, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:18.788359 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59944a2f-174a-492b-99e9-177d4983b749 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:18.857940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1383/2759] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 10:17:18 2022] PUT /identity/v3/projects/6170aac3eb2c4297b4e244f50c007963/users/d2c4d5b3e29a40bf8326ac902aef52f2/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:17:18.873026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2cb0ac-48ed-4440-b373-cb9cba994d59 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:18.873494 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2cb0ac-48ed-4440-b373-cb9cba994d59 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:18.873837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2cb0ac-48ed-4440-b373-cb9cba994d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:18.906338 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8c2cb0ac-48ed-4440-b373-cb9cba994d59 None None] MFA Rules not processed for user `d2c4d5b3e29a40bf8326ac902aef52f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:17:18.909718 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8c2cb0ac-48ed-4440-b373-cb9cba994d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:19.004562 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1377/2760] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:17:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 10:17:19.009062 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59944a2f-174a-492b-99e9-177d4983b749 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:19.070212 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59944a2f-174a-492b-99e9-177d4983b749 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:19.075172 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7389d-9c74-4ffa-9f1d-71ac4a4ae2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:19.075766 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7389d-9c74-4ffa-9f1d-71ac4a4ae2c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:19.076046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7389d-9c74-4ffa-9f1d-71ac4a4ae2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:19.076317 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd7389d-9c74-4ffa-9f1d-71ac4a4ae2c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:19.076951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd7389d-9c74-4ffa-9f1d-71ac4a4ae2c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:19.078862 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd7389d-9c74-4ffa-9f1d-71ac4a4ae2c6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:19.153651 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1384/2761] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 10:17:19 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 10:17:19.158166 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2cb0ac-48ed-4440-b373-cb9cba994d59 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:19.168958 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2cb0ac-48ed-4440-b373-cb9cba994d59 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:19.232757 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61ee37-bc41-4d06-8e2e-09e805b84d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:19.233398 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61ee37-bc41-4d06-8e2e-09e805b84d22 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:19.233681 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61ee37-bc41-4d06-8e2e-09e805b84d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:19.233952 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61ee37-bc41-4d06-8e2e-09e805b84d22 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:19.234592 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d61ee37-bc41-4d06-8e2e-09e805b84d22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:19.237160 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d61ee37-bc41-4d06-8e2e-09e805b84d22 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:19.335796 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1378/2762] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 10:17:19 2022] POST /identity/v3/users => generated 299 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:17:19.339912 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7389d-9c74-4ffa-9f1d-71ac4a4ae2c6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:19.351077 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd7389d-9c74-4ffa-9f1d-71ac4a4ae2c6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:19.355621 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e084d3-e712-4742-8285-26e38a0694bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:19.356207 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e084d3-e712-4742-8285-26e38a0694bf admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:19.356481 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e084d3-e712-4742-8285-26e38a0694bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:19.356779 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e084d3-e712-4742-8285-26e38a0694bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:19.357407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e084d3-e712-4742-8285-26e38a0694bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:19.359033 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e084d3-e712-4742-8285-26e38a0694bf admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:19.370361 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1385/2763] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:17:19.374472 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61ee37-bc41-4d06-8e2e-09e805b84d22 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:19.384461 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61ee37-bc41-4d06-8e2e-09e805b84d22 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:19.389143 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721216f9-cf72-4545-ac19-03b1d4ef13cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:19.389787 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721216f9-cf72-4545-ac19-03b1d4ef13cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:19.390065 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721216f9-cf72-4545-ac19-03b1d4ef13cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:19.390335 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721216f9-cf72-4545-ac19-03b1d4ef13cc admin admin] PATH_INFO: `/v3/projects/b0f88f9ab1c642b5bd2dc7a079e4342c/users/bcb8ae286d3344d1a351744bba07513e/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:19.404639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721216f9-cf72-4545-ac19-03b1d4ef13cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0f88f9ab1c642b5bd2dc7a079e4342c, user_id=bcb8ae286d3344d1a351744bba07513e, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:19.407733 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721216f9-cf72-4545-ac19-03b1d4ef13cc admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:19.502523 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1379/2764] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 10:17:19 2022] PUT /identity/v3/projects/b0f88f9ab1c642b5bd2dc7a079e4342c/users/bcb8ae286d3344d1a351744bba07513e/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:17:19.506470 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e084d3-e712-4742-8285-26e38a0694bf admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:19.517042 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e084d3-e712-4742-8285-26e38a0694bf admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:19.537511 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:17:19.573026 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fceda6-5843-41a6-8aab-d94572dd46cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:19.573633 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fceda6-5843-41a6-8aab-d94572dd46cc admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:19.573914 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fceda6-5843-41a6-8aab-d94572dd46cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:19.574183 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fceda6-5843-41a6-8aab-d94572dd46cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:19.574867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fceda6-5843-41a6-8aab-d94572dd46cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:19.576583 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fceda6-5843-41a6-8aab-d94572dd46cc admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:19.587680 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1386/2765] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:17:19.591880 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721216f9-cf72-4545-ac19-03b1d4ef13cc admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:19.603104 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721216f9-cf72-4545-ac19-03b1d4ef13cc admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:19.607749 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8be0f-2f3b-4e6a-bcc9-f552964a6328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:19.608397 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8be0f-2f3b-4e6a-bcc9-f552964a6328 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:19.608675 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8be0f-2f3b-4e6a-bcc9-f552964a6328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:19.608946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8be0f-2f3b-4e6a-bcc9-f552964a6328 admin admin] PATH_INFO: `/v3/projects/b0f88f9ab1c642b5bd2dc7a079e4342c/users/bcb8ae286d3344d1a351744bba07513e/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:19.611013 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8be0f-2f3b-4e6a-bcc9-f552964a6328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0f88f9ab1c642b5bd2dc7a079e4342c, user_id=bcb8ae286d3344d1a351744bba07513e, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:19.613613 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8be0f-2f3b-4e6a-bcc9-f552964a6328 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:19.624500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1380/2766] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 10:17:19 2022] PUT /identity/v3/projects/b0f88f9ab1c642b5bd2dc7a079e4342c/users/bcb8ae286d3344d1a351744bba07513e/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:17:19.634650 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03da15-1a13-4b79-a992-acf5224c5f62 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:19.634973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03da15-1a13-4b79-a992-acf5224c5f62 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:19.635243 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03da15-1a13-4b79-a992-acf5224c5f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:19.663608 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0c03da15-1a13-4b79-a992-acf5224c5f62 None None] MFA Rules not processed for user `bcb8ae286d3344d1a351744bba07513e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:17:19.664832 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0c03da15-1a13-4b79-a992-acf5224c5f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:19.735573 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1387/2767] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:17:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:17:19.746968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8be0f-2f3b-4e6a-bcc9-f552964a6328 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:19.748819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b1c8be0f-2f3b-4e6a-bcc9-f552964a6328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:19.778211 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:17:19.824299 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8be0f-2f3b-4e6a-bcc9-f552964a6328 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:19.828278 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651cc90-62b3-4edd-841b-868822355431 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:19.829480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651cc90-62b3-4edd-841b-868822355431 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:19.829775 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651cc90-62b3-4edd-841b-868822355431 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:19.830008 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651cc90-62b3-4edd-841b-868822355431 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:19.842081 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3651cc90-62b3-4edd-841b-868822355431 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:19.845476 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3651cc90-62b3-4edd-841b-868822355431 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:19.863650 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1381/2768] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:17:19 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:17:21.134592 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03da15-1a13-4b79-a992-acf5224c5f62 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:21.140300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0c03da15-1a13-4b79-a992-acf5224c5f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:21.150444 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03da15-1a13-4b79-a992-acf5224c5f62 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:21.155016 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5637c44-fb09-4f26-88f8-f92c745683f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:21.155796 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5637c44-fb09-4f26-88f8-f92c745683f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:21.156196 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5637c44-fb09-4f26-88f8-f92c745683f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:21.156542 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5637c44-fb09-4f26-88f8-f92c745683f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:21.229390 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5637c44-fb09-4f26-88f8-f92c745683f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:21.230854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5637c44-fb09-4f26-88f8-f92c745683f4 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:21.240167 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1388/2769] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:17:21 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:17:27.284120 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651cc90-62b3-4edd-841b-868822355431 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:27.292892 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651cc90-62b3-4edd-841b-868822355431 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:27.296790 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc067f64-89a3-45c3-bcd4-51c782e5736d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:27.297400 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc067f64-89a3-45c3-bcd4-51c782e5736d service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:27.297680 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc067f64-89a3-45c3-bcd4-51c782e5736d service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:27.297955 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc067f64-89a3-45c3-bcd4-51c782e5736d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:27.366200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc067f64-89a3-45c3-bcd4-51c782e5736d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:27.368303 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc067f64-89a3-45c3-bcd4-51c782e5736d service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:27.378651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1382/2770] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:17:27 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:17:31.236203 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=80 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:17:31.908774 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5637c44-fb09-4f26-88f8-f92c745683f4 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:31.971193 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5637c44-fb09-4f26-88f8-f92c745683f4 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:31.975949 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db681d8-36b1-41e6-b75c-06988dcc5859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:31.976572 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db681d8-36b1-41e6-b75c-06988dcc5859 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:31.976848 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db681d8-36b1-41e6-b75c-06988dcc5859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:31.977122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db681d8-36b1-41e6-b75c-06988dcc5859 admin admin] PATH_INFO: `/v3/users/aca032f7c5b940b094b86d026a522518` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:31.978222 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db681d8-36b1-41e6-b75c-06988dcc5859 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aca032f7c5b940b094b86d026a522518)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:31.980134 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db681d8-36b1-41e6-b75c-06988dcc5859 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:32.207726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5db681d8-36b1-41e6-b75c-06988dcc5859 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aca032f7c5b940b094b86d026a522518'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:32.214458 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5db681d8-36b1-41e6-b75c-06988dcc5859 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aca032f7c5b940b094b86d026a522518'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:32.323108 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5db681d8-36b1-41e6-b75c-06988dcc5859 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aca032f7c5b940b094b86d026a522518'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:32.323108 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5db681d8-36b1-41e6-b75c-06988dcc5859 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aca032f7c5b940b094b86d026a522518'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:32.333271 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5db681d8-36b1-41e6-b75c-06988dcc5859 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aca032f7c5b940b094b86d026a522518'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:32.339331 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1389/2771] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 10:17:31 2022] DELETE /identity/v3/users/aca032f7c5b940b094b86d026a522518 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:17:32.343903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfdae0-7df9-4689-9b64-2b3142aa2311 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:32.344346 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfdae0-7df9-4689-9b64-2b3142aa2311 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:32.344524 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfdae0-7df9-4689-9b64-2b3142aa2311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:32.364483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-e0bfdae0-7df9-4689-9b64-2b3142aa2311 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:17:32.365672 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e0bfdae0-7df9-4689-9b64-2b3142aa2311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:32.433446 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1383/2772] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:17:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:17:32.443457 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db681d8-36b1-41e6-b75c-06988dcc5859 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:32.447245 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5db681d8-36b1-41e6-b75c-06988dcc5859 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:32.503110 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db681d8-36b1-41e6-b75c-06988dcc5859 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:32.506838 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa424c65-e553-42da-b9bd-b40655ce5925 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:32.507476 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa424c65-e553-42da-b9bd-b40655ce5925 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:32.507760 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa424c65-e553-42da-b9bd-b40655ce5925 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:32.508068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa424c65-e553-42da-b9bd-b40655ce5925 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:32.517866 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa424c65-e553-42da-b9bd-b40655ce5925 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:32.519345 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa424c65-e553-42da-b9bd-b40655ce5925 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:32.528788 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1390/2773] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:17:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:17:32.978139 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfdae0-7df9-4689-9b64-2b3142aa2311 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:32.979599 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e0bfdae0-7df9-4689-9b64-2b3142aa2311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:32.996909 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfdae0-7df9-4689-9b64-2b3142aa2311 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:32.997479 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c69cea-544c-42bb-9e29-8eebf75ee52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:32.998142 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c69cea-544c-42bb-9e29-8eebf75ee52e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:32.998407 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c69cea-544c-42bb-9e29-8eebf75ee52e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:32.998680 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c69cea-544c-42bb-9e29-8eebf75ee52e admin admin] PATH_INFO: `/v3/projects/346b674017524e15b1f47266c683a341` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:32.999860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c69cea-544c-42bb-9e29-8eebf75ee52e admin admin] RBAC: Authorizing `identity:delete_project(project_id=346b674017524e15b1f47266c683a341)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:33.002515 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c69cea-544c-42bb-9e29-8eebf75ee52e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:33.120281 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d8c69cea-544c-42bb-9e29-8eebf75ee52e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '346b674017524e15b1f47266c683a341', 'request_id': 'req-d8c69cea-544c-42bb-9e29-8eebf75ee52e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:33.121441 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:17:33.126320 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d8c69cea-544c-42bb-9e29-8eebf75ee52e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '346b674017524e15b1f47266c683a341', 'request_id': 'req-d8c69cea-544c-42bb-9e29-8eebf75ee52e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:33.133089 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d8c69cea-544c-42bb-9e29-8eebf75ee52e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '346b674017524e15b1f47266c683a341', 'request_id': 'req-d8c69cea-544c-42bb-9e29-8eebf75ee52e'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:33.232456 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d8c69cea-544c-42bb-9e29-8eebf75ee52e admin admin] The token cache is being invalidate because project 346b674017524e15b1f47266c683a341 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 10:17:33.232743 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d8c69cea-544c-42bb-9e29-8eebf75ee52e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:33.233362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d8c69cea-544c-42bb-9e29-8eebf75ee52e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:33.234868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1384/2774] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 10:17:32 2022] DELETE /identity/v3/projects/346b674017524e15b1f47266c683a341 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:17:33.239355 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa424c65-e553-42da-b9bd-b40655ce5925 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:33.248676 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa424c65-e553-42da-b9bd-b40655ce5925 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:33.266662 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:17:33.303006 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d0dc3-6353-4817-bb64-2cbff8a78f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:33.303629 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d0dc3-6353-4817-bb64-2cbff8a78f32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:33.303910 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d0dc3-6353-4817-bb64-2cbff8a78f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:33.304184 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570d0dc3-6353-4817-bb64-2cbff8a78f32 admin admin] PATH_INFO: `/v3/users/d2c4d5b3e29a40bf8326ac902aef52f2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:33.305283 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d0dc3-6353-4817-bb64-2cbff8a78f32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2c4d5b3e29a40bf8326ac902aef52f2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:33.307317 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570d0dc3-6353-4817-bb64-2cbff8a78f32 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:33.522024 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-570d0dc3-6353-4817-bb64-2cbff8a78f32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2c4d5b3e29a40bf8326ac902aef52f2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:33.528692 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-570d0dc3-6353-4817-bb64-2cbff8a78f32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2c4d5b3e29a40bf8326ac902aef52f2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:33.587497 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-570d0dc3-6353-4817-bb64-2cbff8a78f32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2c4d5b3e29a40bf8326ac902aef52f2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:33.588116 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-570d0dc3-6353-4817-bb64-2cbff8a78f32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2c4d5b3e29a40bf8326ac902aef52f2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:33.600502 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-570d0dc3-6353-4817-bb64-2cbff8a78f32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2c4d5b3e29a40bf8326ac902aef52f2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:33.606360 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1391/2775] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 10:17:33 2022] DELETE /identity/v3/users/d2c4d5b3e29a40bf8326ac902aef52f2 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:17:34.006194 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c69cea-544c-42bb-9e29-8eebf75ee52e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:34.007774 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d8c69cea-544c-42bb-9e29-8eebf75ee52e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:34.023663 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c69cea-544c-42bb-9e29-8eebf75ee52e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:34.080413 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93299be1-e6cc-4c1c-90f2-d2c48b495c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:34.081037 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93299be1-e6cc-4c1c-90f2-d2c48b495c41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:34.081310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93299be1-e6cc-4c1c-90f2-d2c48b495c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:34.081578 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93299be1-e6cc-4c1c-90f2-d2c48b495c41 admin admin] PATH_INFO: `/v3/projects/6170aac3eb2c4297b4e244f50c007963` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:34.082722 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93299be1-e6cc-4c1c-90f2-d2c48b495c41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6170aac3eb2c4297b4e244f50c007963)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:34.084580 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93299be1-e6cc-4c1c-90f2-d2c48b495c41 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:34.176385 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-93299be1-e6cc-4c1c-90f2-d2c48b495c41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6170aac3eb2c4297b4e244f50c007963', 'request_id': 'req-93299be1-e6cc-4c1c-90f2-d2c48b495c41'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:34.181837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-93299be1-e6cc-4c1c-90f2-d2c48b495c41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6170aac3eb2c4297b4e244f50c007963', 'request_id': 'req-93299be1-e6cc-4c1c-90f2-d2c48b495c41'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:34.188114 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-93299be1-e6cc-4c1c-90f2-d2c48b495c41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6170aac3eb2c4297b4e244f50c007963', 'request_id': 'req-93299be1-e6cc-4c1c-90f2-d2c48b495c41'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:34.243728 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-93299be1-e6cc-4c1c-90f2-d2c48b495c41 admin admin] The token cache is being invalidate because project 6170aac3eb2c4297b4e244f50c007963 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 10:17:34.244014 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-93299be1-e6cc-4c1c-90f2-d2c48b495c41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:34.244632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-93299be1-e6cc-4c1c-90f2-d2c48b495c41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:34.246117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1385/2776] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 10:17:34 2022] DELETE /identity/v3/projects/6170aac3eb2c4297b4e244f50c007963 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:17:34.250115 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d0dc3-6353-4817-bb64-2cbff8a78f32 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:34.251473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-570d0dc3-6353-4817-bb64-2cbff8a78f32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:34.264583 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570d0dc3-6353-4817-bb64-2cbff8a78f32 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:34.320662 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d39c0-18ff-4d93-ab32-013c9225e06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:34.321282 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d39c0-18ff-4d93-ab32-013c9225e06f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:34.321558 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d39c0-18ff-4d93-ab32-013c9225e06f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:34.321827 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d39c0-18ff-4d93-ab32-013c9225e06f admin admin] PATH_INFO: `/v3/users/bcb8ae286d3344d1a351744bba07513e` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:34.322950 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d39c0-18ff-4d93-ab32-013c9225e06f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcb8ae286d3344d1a351744bba07513e)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:34.324830 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d39c0-18ff-4d93-ab32-013c9225e06f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:34.525200 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:17:34.527847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-595d39c0-18ff-4d93-ab32-013c9225e06f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcb8ae286d3344d1a351744bba07513e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:34.535610 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-595d39c0-18ff-4d93-ab32-013c9225e06f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcb8ae286d3344d1a351744bba07513e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:34.628059 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-595d39c0-18ff-4d93-ab32-013c9225e06f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcb8ae286d3344d1a351744bba07513e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:34.628692 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-595d39c0-18ff-4d93-ab32-013c9225e06f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcb8ae286d3344d1a351744bba07513e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:34.641211 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-595d39c0-18ff-4d93-ab32-013c9225e06f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcb8ae286d3344d1a351744bba07513e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:34.647047 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1392/2777] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 10:17:34 2022] DELETE /identity/v3/users/bcb8ae286d3344d1a351744bba07513e => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:17:35.075104 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93299be1-e6cc-4c1c-90f2-d2c48b495c41 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:35.077704 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-93299be1-e6cc-4c1c-90f2-d2c48b495c41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:35.091466 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93299be1-e6cc-4c1c-90f2-d2c48b495c41 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:35.150891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f85ba-6b32-4b8f-803a-f92f00054965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:35.151530 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f85ba-6b32-4b8f-803a-f92f00054965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:35.151823 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f85ba-6b32-4b8f-803a-f92f00054965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:35.152117 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f85ba-6b32-4b8f-803a-f92f00054965 admin admin] PATH_INFO: `/v3/projects/b0f88f9ab1c642b5bd2dc7a079e4342c` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:35.153233 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f85ba-6b32-4b8f-803a-f92f00054965 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0f88f9ab1c642b5bd2dc7a079e4342c)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:35.155250 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f85ba-6b32-4b8f-803a-f92f00054965 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:35.166821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:17:35.251097 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0c7f85ba-6b32-4b8f-803a-f92f00054965 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0f88f9ab1c642b5bd2dc7a079e4342c', 'request_id': 'req-0c7f85ba-6b32-4b8f-803a-f92f00054965'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:35.256704 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0c7f85ba-6b32-4b8f-803a-f92f00054965 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0f88f9ab1c642b5bd2dc7a079e4342c', 'request_id': 'req-0c7f85ba-6b32-4b8f-803a-f92f00054965'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:35.262855 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0c7f85ba-6b32-4b8f-803a-f92f00054965 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0f88f9ab1c642b5bd2dc7a079e4342c', 'request_id': 'req-0c7f85ba-6b32-4b8f-803a-f92f00054965'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:35.318453 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0c7f85ba-6b32-4b8f-803a-f92f00054965 admin admin] The token cache is being invalidate because project b0f88f9ab1c642b5bd2dc7a079e4342c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 10:17:35.318759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0c7f85ba-6b32-4b8f-803a-f92f00054965 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:35.319414 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0c7f85ba-6b32-4b8f-803a-f92f00054965 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:35.320895 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1386/2778] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 10:17:35 2022] DELETE /identity/v3/projects/b0f88f9ab1c642b5bd2dc7a079e4342c => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:17:35.328135 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9983ca-9b79-4447-8706-96e237f1172c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:35.328367 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9983ca-9b79-4447-8706-96e237f1172c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:35.328647 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9983ca-9b79-4447-8706-96e237f1172c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:35.349820 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8a9983ca-9b79-4447-8706-96e237f1172c None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:17:35.351061 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8a9983ca-9b79-4447-8706-96e237f1172c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:35.410370 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1393/2779] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:17:35.420334 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d3f4e-c902-4058-9d68-d13061bb3413 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:35.420669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d3f4e-c902-4058-9d68-d13061bb3413 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:35.420945 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d3f4e-c902-4058-9d68-d13061bb3413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:35.440869 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-9a1d3f4e-c902-4058-9d68-d13061bb3413 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:17:35.442071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-9a1d3f4e-c902-4058-9d68-d13061bb3413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:35.450891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1387/2780] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:17:35.456280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9983ca-9b79-4447-8706-96e237f1172c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:35.465961 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9983ca-9b79-4447-8706-96e237f1172c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:35.471466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b9aaf-dccd-4e71-8d86-f35398825525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:35.472072 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b9aaf-dccd-4e71-8d86-f35398825525 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:35.472355 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b9aaf-dccd-4e71-8d86-f35398825525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:35.472623 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b9aaf-dccd-4e71-8d86-f35398825525 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:35.473303 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b9aaf-dccd-4e71-8d86-f35398825525 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 10:17:35.473616 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b9aaf-dccd-4e71-8d86-f35398825525 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:35.475533 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b9aaf-dccd-4e71-8d86-f35398825525 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:35.488221 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1394/2781] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 10:17:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 10:17:35.492233 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d3f4e-c902-4058-9d68-d13061bb3413 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:35.501913 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d3f4e-c902-4058-9d68-d13061bb3413 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:35.506525 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b697b-8fd1-4c1d-a1a0-2f1ba537f3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:35.507155 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b697b-8fd1-4c1d-a1a0-2f1ba537f3bf admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:35.507434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b697b-8fd1-4c1d-a1a0-2f1ba537f3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:35.507701 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b697b-8fd1-4c1d-a1a0-2f1ba537f3bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:35.508352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b697b-8fd1-4c1d-a1a0-2f1ba537f3bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:35.510233 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0b697b-8fd1-4c1d-a1a0-2f1ba537f3bf admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:35.596518 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1388/2782] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 10:17:35 2022] POST /identity/v3/projects => generated 401 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:17:35.601140 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b9aaf-dccd-4e71-8d86-f35398825525 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:35.611600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b9aaf-dccd-4e71-8d86-f35398825525 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:35.668472 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d827b-8b0c-4463-952c-fa9bccb9611b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:35.669082 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d827b-8b0c-4463-952c-fa9bccb9611b admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:35.669360 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d827b-8b0c-4463-952c-fa9bccb9611b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:35.669627 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d827b-8b0c-4463-952c-fa9bccb9611b admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:35.670282 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d827b-8b0c-4463-952c-fa9bccb9611b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:35.672196 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d827b-8b0c-4463-952c-fa9bccb9611b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:35.771066 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1395/2783] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 10:17:35 2022] POST /identity/v3/users => generated 301 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:17:35.775314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b697b-8fd1-4c1d-a1a0-2f1ba537f3bf admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:35.785673 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b697b-8fd1-4c1d-a1a0-2f1ba537f3bf admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:35.790243 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086cdf5-6753-4011-a357-d32c23e78c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:35.790889 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086cdf5-6753-4011-a357-d32c23e78c56 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:35.791172 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086cdf5-6753-4011-a357-d32c23e78c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:35.791445 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086cdf5-6753-4011-a357-d32c23e78c56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:35.792094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086cdf5-6753-4011-a357-d32c23e78c56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:35.793666 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086cdf5-6753-4011-a357-d32c23e78c56 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:35.803589 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1389/2784] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:17:35.809601 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d827b-8b0c-4463-952c-fa9bccb9611b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:35.820311 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d827b-8b0c-4463-952c-fa9bccb9611b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:35.825434 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ddc0b4-fc90-46e5-a87b-ee7bf01c7de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:35.826064 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ddc0b4-fc90-46e5-a87b-ee7bf01c7de1 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:35.826347 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ddc0b4-fc90-46e5-a87b-ee7bf01c7de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:35.826618 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ddc0b4-fc90-46e5-a87b-ee7bf01c7de1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:35.827297 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ddc0b4-fc90-46e5-a87b-ee7bf01c7de1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:35.828874 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ddc0b4-fc90-46e5-a87b-ee7bf01c7de1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:35.843231 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1396/2785] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:17:35.846825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086cdf5-6753-4011-a357-d32c23e78c56 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:35.857765 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086cdf5-6753-4011-a357-d32c23e78c56 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:35.862572 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778547ec-472b-4b39-954a-be1992268f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:35.863292 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778547ec-472b-4b39-954a-be1992268f05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:35.863574 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778547ec-472b-4b39-954a-be1992268f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:35.863849 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778547ec-472b-4b39-954a-be1992268f05 admin admin] PATH_INFO: `/v3/projects/1771a73604984905b912ff8c70d924e4/users/56fd1f8f2ce443308ec8ca51c1b9c515/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:35.879717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778547ec-472b-4b39-954a-be1992268f05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1771a73604984905b912ff8c70d924e4, user_id=56fd1f8f2ce443308ec8ca51c1b9c515, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:35.882357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778547ec-472b-4b39-954a-be1992268f05 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:35.953526 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1390/2786] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 10:17:35 2022] PUT /identity/v3/projects/1771a73604984905b912ff8c70d924e4/users/56fd1f8f2ce443308ec8ca51c1b9c515/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:17:35.965674 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd87cf7-b8ac-4209-979b-3da0d739411a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:35.965996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd87cf7-b8ac-4209-979b-3da0d739411a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:35.966270 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd87cf7-b8ac-4209-979b-3da0d739411a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:35.994410 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-7bd87cf7-b8ac-4209-979b-3da0d739411a None None] MFA Rules not processed for user `56fd1f8f2ce443308ec8ca51c1b9c515`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:17:35.995725 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7bd87cf7-b8ac-4209-979b-3da0d739411a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:36.011751 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:17:36.088322 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1397/2787] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:17:35 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:17:36.093235 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778547ec-472b-4b39-954a-be1992268f05 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:36.180156 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778547ec-472b-4b39-954a-be1992268f05 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:36.184658 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8a563-7dcd-4f7c-ada9-a77eae187f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:36.185268 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8a563-7dcd-4f7c-ada9-a77eae187f48 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:36.185549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8a563-7dcd-4f7c-ada9-a77eae187f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:36.185825 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8a563-7dcd-4f7c-ada9-a77eae187f48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:36.186448 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8a563-7dcd-4f7c-ada9-a77eae187f48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:36.188590 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8a563-7dcd-4f7c-ada9-a77eae187f48 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:36.268010 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1391/2788] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 10:17:36 2022] POST /identity/v3/projects => generated 401 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 10:17:36.272204 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd87cf7-b8ac-4209-979b-3da0d739411a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:36.282910 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd87cf7-b8ac-4209-979b-3da0d739411a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:36.338534 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd503674-35a2-45b8-9d71-b5b980ee450f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:36.339162 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd503674-35a2-45b8-9d71-b5b980ee450f admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:36.339466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd503674-35a2-45b8-9d71-b5b980ee450f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:36.339739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd503674-35a2-45b8-9d71-b5b980ee450f admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:36.340363 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd503674-35a2-45b8-9d71-b5b980ee450f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:36.342245 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd503674-35a2-45b8-9d71-b5b980ee450f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:36.428280 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1398/2789] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 10:17:36 2022] POST /identity/v3/users => generated 307 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:17:36.432204 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8a563-7dcd-4f7c-ada9-a77eae187f48 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:36.442588 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8a563-7dcd-4f7c-ada9-a77eae187f48 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:36.447212 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff46f89-3f4c-473e-814a-aaa01c1210da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:36.447801 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff46f89-3f4c-473e-814a-aaa01c1210da admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:36.448078 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff46f89-3f4c-473e-814a-aaa01c1210da admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:36.448344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff46f89-3f4c-473e-814a-aaa01c1210da admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:36.448979 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff46f89-3f4c-473e-814a-aaa01c1210da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:36.450577 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff46f89-3f4c-473e-814a-aaa01c1210da admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:36.460905 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1392/2790] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:17:36.467079 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd503674-35a2-45b8-9d71-b5b980ee450f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:36.477870 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd503674-35a2-45b8-9d71-b5b980ee450f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:36.482469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72027fd4-be04-4d4f-a0e5-312024da0808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:36.483145 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72027fd4-be04-4d4f-a0e5-312024da0808 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:36.483428 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72027fd4-be04-4d4f-a0e5-312024da0808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:36.483705 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72027fd4-be04-4d4f-a0e5-312024da0808 admin admin] PATH_INFO: `/v3/projects/9505b39511da4a61866abb932e1875dd/users/aa9ca42530df4c17b3ab2978e85b664c/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:36.500413 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72027fd4-be04-4d4f-a0e5-312024da0808 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9505b39511da4a61866abb932e1875dd, user_id=aa9ca42530df4c17b3ab2978e85b664c, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:36.503318 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72027fd4-be04-4d4f-a0e5-312024da0808 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:36.645209 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1399/2791] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 10:17:36 2022] PUT /identity/v3/projects/9505b39511da4a61866abb932e1875dd/users/aa9ca42530df4c17b3ab2978e85b664c/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:17:36.649206 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff46f89-3f4c-473e-814a-aaa01c1210da admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:36.659830 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff46f89-3f4c-473e-814a-aaa01c1210da admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:36.664479 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:17:36.717500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac91eef-64d3-45a9-9e95-12986a635822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:36.718265 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac91eef-64d3-45a9-9e95-12986a635822 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:36.718546 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac91eef-64d3-45a9-9e95-12986a635822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:36.718854 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac91eef-64d3-45a9-9e95-12986a635822 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:36.719504 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac91eef-64d3-45a9-9e95-12986a635822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:36.721597 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac91eef-64d3-45a9-9e95-12986a635822 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:36.733197 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1393/2792] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:17:36.737996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72027fd4-be04-4d4f-a0e5-312024da0808 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:36.748337 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72027fd4-be04-4d4f-a0e5-312024da0808 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:36.752862 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e9761-20ae-490a-bc73-ef0ff55714ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:36.753501 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e9761-20ae-490a-bc73-ef0ff55714ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:36.753776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e9761-20ae-490a-bc73-ef0ff55714ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:36.754045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e9761-20ae-490a-bc73-ef0ff55714ba admin admin] PATH_INFO: `/v3/projects/9505b39511da4a61866abb932e1875dd/users/aa9ca42530df4c17b3ab2978e85b664c/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:36.755973 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e9761-20ae-490a-bc73-ef0ff55714ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=9505b39511da4a61866abb932e1875dd, user_id=aa9ca42530df4c17b3ab2978e85b664c, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:36.758663 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e9761-20ae-490a-bc73-ef0ff55714ba admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:36.769467 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1400/2793] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 10:17:36 2022] PUT /identity/v3/projects/9505b39511da4a61866abb932e1875dd/users/aa9ca42530df4c17b3ab2978e85b664c/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:17:36.780364 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9196e74-c175-4985-81a4-7ef8ca467540 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:36.780645 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9196e74-c175-4985-81a4-7ef8ca467540 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:36.780920 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9196e74-c175-4985-81a4-7ef8ca467540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:36.809822 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-c9196e74-c175-4985-81a4-7ef8ca467540 None None] MFA Rules not processed for user `aa9ca42530df4c17b3ab2978e85b664c`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:17:36.811075 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-c9196e74-c175-4985-81a4-7ef8ca467540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:36.881114 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1394/2794] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:17:36 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:17:36.892715 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e9761-20ae-490a-bc73-ef0ff55714ba admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:36.894457 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c75e9761-20ae-490a-bc73-ef0ff55714ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:36.951410 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e9761-20ae-490a-bc73-ef0ff55714ba admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:36.955500 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98207fbb-b3ea-403e-8926-00530ee6802f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:36.956126 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98207fbb-b3ea-403e-8926-00530ee6802f service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:36.956572 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98207fbb-b3ea-403e-8926-00530ee6802f service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:36.956692 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98207fbb-b3ea-403e-8926-00530ee6802f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:36.966316 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98207fbb-b3ea-403e-8926-00530ee6802f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:36.967815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98207fbb-b3ea-403e-8926-00530ee6802f service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:36.978347 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1401/2795] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:17:36 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:17:37.163229 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9196e74-c175-4985-81a4-7ef8ca467540 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:37.172733 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9196e74-c175-4985-81a4-7ef8ca467540 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:37.178164 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18facbf9-3b4a-4402-bd55-6309b52643a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:37.178807 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18facbf9-3b4a-4402-bd55-6309b52643a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:37.179090 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18facbf9-3b4a-4402-bd55-6309b52643a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:37.179364 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18facbf9-3b4a-4402-bd55-6309b52643a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:37.237000 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18facbf9-3b4a-4402-bd55-6309b52643a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:37.238353 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18facbf9-3b4a-4402-bd55-6309b52643a3 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:37.247287 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1395/2796] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:17:37 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:17:43.522421 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98207fbb-b3ea-403e-8926-00530ee6802f service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:43.535506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:17:43.589852 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98207fbb-b3ea-403e-8926-00530ee6802f service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:43.594640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1200b23a-7469-4f03-a57a-a3bbfb061547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:43.595409 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200b23a-7469-4f03-a57a-a3bbfb061547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:43.595698 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200b23a-7469-4f03-a57a-a3bbfb061547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:43.596749 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200b23a-7469-4f03-a57a-a3bbfb061547 admin admin] PATH_INFO: `/v3/users/56fd1f8f2ce443308ec8ca51c1b9c515` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:43.598079 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1200b23a-7469-4f03-a57a-a3bbfb061547 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56fd1f8f2ce443308ec8ca51c1b9c515)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:43.600275 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1200b23a-7469-4f03-a57a-a3bbfb061547 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:43.866630 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1200b23a-7469-4f03-a57a-a3bbfb061547 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56fd1f8f2ce443308ec8ca51c1b9c515'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:43.874691 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1200b23a-7469-4f03-a57a-a3bbfb061547 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56fd1f8f2ce443308ec8ca51c1b9c515'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:43.964998 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1200b23a-7469-4f03-a57a-a3bbfb061547 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56fd1f8f2ce443308ec8ca51c1b9c515'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:43.965710 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1200b23a-7469-4f03-a57a-a3bbfb061547 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56fd1f8f2ce443308ec8ca51c1b9c515'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:43.978596 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1200b23a-7469-4f03-a57a-a3bbfb061547 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56fd1f8f2ce443308ec8ca51c1b9c515'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:43.984786 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1402/2797] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 10:17:43 2022] DELETE /identity/v3/users/56fd1f8f2ce443308ec8ca51c1b9c515 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:17:43.989332 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0e0e4-2b11-4485-8bdf-55b7e370d176 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:43.989824 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0e0e4-2b11-4485-8bdf-55b7e370d176 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:43.989913 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0e0e4-2b11-4485-8bdf-55b7e370d176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:44.009689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-52e0e0e4-2b11-4485-8bdf-55b7e370d176 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:17:44.010916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-52e0e0e4-2b11-4485-8bdf-55b7e370d176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:44.046759 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=100 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:17:44.085406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1396/2798] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:17:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:17:44.095974 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1200b23a-7469-4f03-a57a-a3bbfb061547 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:44.097400 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1200b23a-7469-4f03-a57a-a3bbfb061547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:44.180846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1200b23a-7469-4f03-a57a-a3bbfb061547 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:44.184906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ee971-7418-4fd5-8755-55d9b106c533 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:44.185632 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ee971-7418-4fd5-8755-55d9b106c533 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:44.185871 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ee971-7418-4fd5-8755-55d9b106c533 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:44.186190 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ee971-7418-4fd5-8755-55d9b106c533 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:44.197422 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ee971-7418-4fd5-8755-55d9b106c533 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:44.198905 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ee971-7418-4fd5-8755-55d9b106c533 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:44.210698 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1403/2799] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:17:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:17:44.658078 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0e0e4-2b11-4485-8bdf-55b7e370d176 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:44.660418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-52e0e0e4-2b11-4485-8bdf-55b7e370d176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:44.678821 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0e0e4-2b11-4485-8bdf-55b7e370d176 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:44.679379 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3efd6-d2ff-4ecf-946c-dc429542275a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:44.680240 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3efd6-d2ff-4ecf-946c-dc429542275a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:44.680454 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3efd6-d2ff-4ecf-946c-dc429542275a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:44.680794 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3efd6-d2ff-4ecf-946c-dc429542275a admin admin] PATH_INFO: `/v3/projects/1771a73604984905b912ff8c70d924e4` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:44.682065 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3efd6-d2ff-4ecf-946c-dc429542275a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1771a73604984905b912ff8c70d924e4)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:44.684550 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c3efd6-d2ff-4ecf-946c-dc429542275a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:44.823030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-26c3efd6-d2ff-4ecf-946c-dc429542275a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1771a73604984905b912ff8c70d924e4', 'request_id': 'req-26c3efd6-d2ff-4ecf-946c-dc429542275a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:44.828865 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-26c3efd6-d2ff-4ecf-946c-dc429542275a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1771a73604984905b912ff8c70d924e4', 'request_id': 'req-26c3efd6-d2ff-4ecf-946c-dc429542275a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:44.836733 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-26c3efd6-d2ff-4ecf-946c-dc429542275a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1771a73604984905b912ff8c70d924e4', 'request_id': 'req-26c3efd6-d2ff-4ecf-946c-dc429542275a'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:44.914000 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-26c3efd6-d2ff-4ecf-946c-dc429542275a admin admin] The token cache is being invalidate because project 1771a73604984905b912ff8c70d924e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 10:17:44.914298 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-26c3efd6-d2ff-4ecf-946c-dc429542275a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:44.915048 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-26c3efd6-d2ff-4ecf-946c-dc429542275a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:44.916639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1397/2800] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 10:17:44 2022] DELETE /identity/v3/projects/1771a73604984905b912ff8c70d924e4 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:17:44.925772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ee971-7418-4fd5-8755-55d9b106c533 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:44.935719 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ee971-7418-4fd5-8755-55d9b106c533 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:44.968509 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:17:44.996100 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7812a218-64fa-487e-b3b8-2496fa924e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:44.996733 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812a218-64fa-487e-b3b8-2496fa924e09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:44.997031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812a218-64fa-487e-b3b8-2496fa924e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:44.997385 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812a218-64fa-487e-b3b8-2496fa924e09 admin admin] PATH_INFO: `/v3/users/aa9ca42530df4c17b3ab2978e85b664c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:44.998650 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7812a218-64fa-487e-b3b8-2496fa924e09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa9ca42530df4c17b3ab2978e85b664c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:45.000627 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7812a218-64fa-487e-b3b8-2496fa924e09 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:45.246679 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7812a218-64fa-487e-b3b8-2496fa924e09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa9ca42530df4c17b3ab2978e85b664c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:45.253579 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7812a218-64fa-487e-b3b8-2496fa924e09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa9ca42530df4c17b3ab2978e85b664c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:45.329564 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7812a218-64fa-487e-b3b8-2496fa924e09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa9ca42530df4c17b3ab2978e85b664c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:45.330778 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7812a218-64fa-487e-b3b8-2496fa924e09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa9ca42530df4c17b3ab2978e85b664c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:45.344423 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7812a218-64fa-487e-b3b8-2496fa924e09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa9ca42530df4c17b3ab2978e85b664c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:45.351214 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1404/2801] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 10:17:44 2022] DELETE /identity/v3/users/aa9ca42530df4c17b3ab2978e85b664c => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 10:17:46.108854 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3efd6-d2ff-4ecf-946c-dc429542275a admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:46.110349 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-26c3efd6-d2ff-4ecf-946c-dc429542275a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:46.127052 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3efd6-d2ff-4ecf-946c-dc429542275a admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:46.194392 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc85585b-f4ae-4c47-b32d-941b94de392d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:46.195062 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85585b-f4ae-4c47-b32d-941b94de392d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:46.195345 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85585b-f4ae-4c47-b32d-941b94de392d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:46.195614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc85585b-f4ae-4c47-b32d-941b94de392d admin admin] PATH_INFO: `/v3/projects/9505b39511da4a61866abb932e1875dd` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:46.196745 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc85585b-f4ae-4c47-b32d-941b94de392d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9505b39511da4a61866abb932e1875dd)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:46.198632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc85585b-f4ae-4c47-b32d-941b94de392d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:46.320994 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-bc85585b-f4ae-4c47-b32d-941b94de392d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9505b39511da4a61866abb932e1875dd', 'request_id': 'req-bc85585b-f4ae-4c47-b32d-941b94de392d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:46.325050 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:17:46.328088 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-bc85585b-f4ae-4c47-b32d-941b94de392d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9505b39511da4a61866abb932e1875dd', 'request_id': 'req-bc85585b-f4ae-4c47-b32d-941b94de392d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:46.335923 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-bc85585b-f4ae-4c47-b32d-941b94de392d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9505b39511da4a61866abb932e1875dd', 'request_id': 'req-bc85585b-f4ae-4c47-b32d-941b94de392d'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:46.489830 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-bc85585b-f4ae-4c47-b32d-941b94de392d admin admin] The token cache is being invalidate because project 9505b39511da4a61866abb932e1875dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 10:17:46.490119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-bc85585b-f4ae-4c47-b32d-941b94de392d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:46.490788 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-bc85585b-f4ae-4c47-b32d-941b94de392d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:46.492353 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1398/2802] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 10:17:46 2022] DELETE /identity/v3/projects/9505b39511da4a61866abb932e1875dd => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:17:46.497462 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46013518-540e-49a2-ad94-acccfeec0c26 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:46.497953 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46013518-540e-49a2-ad94-acccfeec0c26 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:46.498043 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46013518-540e-49a2-ad94-acccfeec0c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:46.518572 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-46013518-540e-49a2-ad94-acccfeec0c26 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:17:46.519830 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-46013518-540e-49a2-ad94-acccfeec0c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:46.585587 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1405/2803] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:17:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:17:46.593455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa2ed5-ab22-4146-be24-73147fa0ed3c None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:46.593760 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa2ed5-ab22-4146-be24-73147fa0ed3c None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:46.594040 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa2ed5-ab22-4146-be24-73147fa0ed3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:46.616047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-25aa2ed5-ab22-4146-be24-73147fa0ed3c None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:17:46.617368 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-25aa2ed5-ab22-4146-be24-73147fa0ed3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:46.625897 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1399/2804] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:17:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 10:17:46.630383 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46013518-540e-49a2-ad94-acccfeec0c26 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:46.640984 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46013518-540e-49a2-ad94-acccfeec0c26 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:46.646996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa974f1-62ad-4390-b025-63568c542446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:46.656022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa974f1-62ad-4390-b025-63568c542446 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:46.656022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa974f1-62ad-4390-b025-63568c542446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:46.656022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa974f1-62ad-4390-b025-63568c542446 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:46.656022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa974f1-62ad-4390-b025-63568c542446 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 10:17:46.656022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa974f1-62ad-4390-b025-63568c542446 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:46.656022 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa974f1-62ad-4390-b025-63568c542446 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:46.664522 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1406/2805] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 10:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 10:17:46.669971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa2ed5-ab22-4146-be24-73147fa0ed3c None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:46.679340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa2ed5-ab22-4146-be24-73147fa0ed3c None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:46.686035 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa556ec7-8964-4b8f-9270-0adf96443dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:46.686649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa556ec7-8964-4b8f-9270-0adf96443dae admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:46.686964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa556ec7-8964-4b8f-9270-0adf96443dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:46.687258 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa556ec7-8964-4b8f-9270-0adf96443dae admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:46.687912 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa556ec7-8964-4b8f-9270-0adf96443dae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:46.698103 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa556ec7-8964-4b8f-9270-0adf96443dae admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:46.805917 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1400/2806] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 10:17:46 2022] POST /identity/v3/projects => generated 409 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:17:46.810200 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa974f1-62ad-4390-b025-63568c542446 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:46.820923 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa974f1-62ad-4390-b025-63568c542446 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:46.862829 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:17:46.881384 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c20f5f-3cab-4f55-90ba-bf4baf963601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:46.882023 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c20f5f-3cab-4f55-90ba-bf4baf963601 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:46.882305 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c20f5f-3cab-4f55-90ba-bf4baf963601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:46.882665 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c20f5f-3cab-4f55-90ba-bf4baf963601 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:46.883326 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c20f5f-3cab-4f55-90ba-bf4baf963601 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:46.885217 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c20f5f-3cab-4f55-90ba-bf4baf963601 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:46.985074 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1407/2807] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 10:17:46 2022] POST /identity/v3/users => generated 305 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:17:46.986354 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa556ec7-8964-4b8f-9270-0adf96443dae admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:47.003128 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa556ec7-8964-4b8f-9270-0adf96443dae admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:47.007941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b1ee7e-3d1c-4645-9451-c1cfed853b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:47.008688 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1ee7e-3d1c-4645-9451-c1cfed853b91 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:47.008886 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1ee7e-3d1c-4645-9451-c1cfed853b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:47.009162 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b1ee7e-3d1c-4645-9451-c1cfed853b91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:47.009812 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b1ee7e-3d1c-4645-9451-c1cfed853b91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:47.011594 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b1ee7e-3d1c-4645-9451-c1cfed853b91 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:47.022569 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1401/2808] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:17:47.026698 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c20f5f-3cab-4f55-90ba-bf4baf963601 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:47.036941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c20f5f-3cab-4f55-90ba-bf4baf963601 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:47.041749 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9a19e-3074-4c6c-bf43-9f421d8b123f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:47.046454 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9a19e-3074-4c6c-bf43-9f421d8b123f admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:47.046454 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9a19e-3074-4c6c-bf43-9f421d8b123f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:47.046454 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9a19e-3074-4c6c-bf43-9f421d8b123f admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:47.046454 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9a19e-3074-4c6c-bf43-9f421d8b123f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:47.046454 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9a19e-3074-4c6c-bf43-9f421d8b123f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:47.057149 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1408/2809] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:17:47.062374 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b1ee7e-3d1c-4645-9451-c1cfed853b91 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:47.074638 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b1ee7e-3d1c-4645-9451-c1cfed853b91 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:47.081167 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a3bfe-95b6-4749-b29c-9681c48e553d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:47.081931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a3bfe-95b6-4749-b29c-9681c48e553d admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:47.082251 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a3bfe-95b6-4749-b29c-9681c48e553d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:47.082566 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a3bfe-95b6-4749-b29c-9681c48e553d admin admin] PATH_INFO: `/v3/projects/bbfbe0266fba47eaa732281a886df41d/users/319cda3819914949ae46ff5d38303105/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:47.098393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a3bfe-95b6-4749-b29c-9681c48e553d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbfbe0266fba47eaa732281a886df41d, user_id=319cda3819914949ae46ff5d38303105, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:47.101309 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a3bfe-95b6-4749-b29c-9681c48e553d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:47.198523 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1402/2810] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 10:17:47 2022] PUT /identity/v3/projects/bbfbe0266fba47eaa732281a886df41d/users/319cda3819914949ae46ff5d38303105/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:17:47.211117 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662eb0f6-bd98-476b-bcb6-38cad0e2f511 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:47.211407 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662eb0f6-bd98-476b-bcb6-38cad0e2f511 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:47.211707 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662eb0f6-bd98-476b-bcb6-38cad0e2f511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:47.241098 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-662eb0f6-bd98-476b-bcb6-38cad0e2f511 None None] MFA Rules not processed for user `319cda3819914949ae46ff5d38303105`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:17:47.242322 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-662eb0f6-bd98-476b-bcb6-38cad0e2f511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:47.312456 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1409/2811] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:17:47 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:17:47.317863 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a3bfe-95b6-4749-b29c-9681c48e553d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:47.384998 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a3bfe-95b6-4749-b29c-9681c48e553d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:47.389787 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ebe24d-ee10-463b-bd52-95a6f303a582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:47.390406 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebe24d-ee10-463b-bd52-95a6f303a582 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:47.390684 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebe24d-ee10-463b-bd52-95a6f303a582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:47.390976 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ebe24d-ee10-463b-bd52-95a6f303a582 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:47.391609 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ebe24d-ee10-463b-bd52-95a6f303a582 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:47.393562 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ebe24d-ee10-463b-bd52-95a6f303a582 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:47.510497 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1403/2812] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 10:17:47 2022] POST /identity/v3/projects => generated 409 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:17:47.514661 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662eb0f6-bd98-476b-bcb6-38cad0e2f511 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:47.525707 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662eb0f6-bd98-476b-bcb6-38cad0e2f511 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:47.587396 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de10b25-7fd4-42a4-b4af-b06b5196d4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:47.588005 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de10b25-7fd4-42a4-b4af-b06b5196d4d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:47.588286 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de10b25-7fd4-42a4-b4af-b06b5196d4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:47.588561 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de10b25-7fd4-42a4-b4af-b06b5196d4d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:47.589196 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de10b25-7fd4-42a4-b4af-b06b5196d4d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:47.591147 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de10b25-7fd4-42a4-b4af-b06b5196d4d0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:47.695331 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1410/2813] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 10:17:47 2022] POST /identity/v3/users => generated 311 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:17:47.699324 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ebe24d-ee10-463b-bd52-95a6f303a582 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:47.709768 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ebe24d-ee10-463b-bd52-95a6f303a582 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:47.714542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509bb3c-5aca-409e-9ccc-19f866491502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:47.715315 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509bb3c-5aca-409e-9ccc-19f866491502 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:47.715589 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509bb3c-5aca-409e-9ccc-19f866491502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:47.715864 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509bb3c-5aca-409e-9ccc-19f866491502 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:47.716605 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8509bb3c-5aca-409e-9ccc-19f866491502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:47.718193 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8509bb3c-5aca-409e-9ccc-19f866491502 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:47.730852 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1404/2814] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:17:47.737272 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de10b25-7fd4-42a4-b4af-b06b5196d4d0 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:47.749032 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de10b25-7fd4-42a4-b4af-b06b5196d4d0 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:47.753909 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90938d82-0eae-48c8-8ced-ae72da6eeaf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:47.754584 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90938d82-0eae-48c8-8ced-ae72da6eeaf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:47.754900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90938d82-0eae-48c8-8ced-ae72da6eeaf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:47.755182 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90938d82-0eae-48c8-8ced-ae72da6eeaf8 admin admin] PATH_INFO: `/v3/projects/afc76ac80d5046a3a31be97a59197aa7/users/95226d62d81140189cbd84fa3555bc87/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:47.772652 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90938d82-0eae-48c8-8ced-ae72da6eeaf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afc76ac80d5046a3a31be97a59197aa7, user_id=95226d62d81140189cbd84fa3555bc87, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:47.775379 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90938d82-0eae-48c8-8ced-ae72da6eeaf8 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:47.868283 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1411/2815] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 10:17:47 2022] PUT /identity/v3/projects/afc76ac80d5046a3a31be97a59197aa7/users/95226d62d81140189cbd84fa3555bc87/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 10:17:47.872693 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509bb3c-5aca-409e-9ccc-19f866491502 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:47.883111 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509bb3c-5aca-409e-9ccc-19f866491502 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:47.931402 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:17:47.940515 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7906dd-4e52-4c2f-ba6e-5cf9a658e87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:47.941152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7906dd-4e52-4c2f-ba6e-5cf9a658e87a admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:47.941435 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7906dd-4e52-4c2f-ba6e-5cf9a658e87a admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:47.945410 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7906dd-4e52-4c2f-ba6e-5cf9a658e87a admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:47.945410 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7906dd-4e52-4c2f-ba6e-5cf9a658e87a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:47.945410 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7906dd-4e52-4c2f-ba6e-5cf9a658e87a admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:47.953810 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1405/2816] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:17:47.958627 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90938d82-0eae-48c8-8ced-ae72da6eeaf8 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:47.969072 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90938d82-0eae-48c8-8ced-ae72da6eeaf8 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:47.974629 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c210a0c-2bd1-4910-83b3-38a06aed2a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:47.975309 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c210a0c-2bd1-4910-83b3-38a06aed2a32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:47.975591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c210a0c-2bd1-4910-83b3-38a06aed2a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:47.975867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c210a0c-2bd1-4910-83b3-38a06aed2a32 admin admin] PATH_INFO: `/v3/projects/afc76ac80d5046a3a31be97a59197aa7/users/95226d62d81140189cbd84fa3555bc87/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:47.979774 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c210a0c-2bd1-4910-83b3-38a06aed2a32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afc76ac80d5046a3a31be97a59197aa7, user_id=95226d62d81140189cbd84fa3555bc87, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:47.980669 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c210a0c-2bd1-4910-83b3-38a06aed2a32 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:47.987030 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:17:47.991420 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1412/2817] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 10:17:47 2022] PUT /identity/v3/projects/afc76ac80d5046a3a31be97a59197aa7/users/95226d62d81140189cbd84fa3555bc87/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:17:48.002566 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d8875-1e1f-474d-a5b1-fac91060b0ef None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:48.002883 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d8875-1e1f-474d-a5b1-fac91060b0ef None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:48.003155 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d8875-1e1f-474d-a5b1-fac91060b0ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:48.032441 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-6d1d8875-1e1f-474d-a5b1-fac91060b0ef None None] MFA Rules not processed for user `95226d62d81140189cbd84fa3555bc87`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:17:48.039026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6d1d8875-1e1f-474d-a5b1-fac91060b0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:48.114149 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1406/2818] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:17:48 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:17:48.128474 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c210a0c-2bd1-4910-83b3-38a06aed2a32 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:48.130628 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4c210a0c-2bd1-4910-83b3-38a06aed2a32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:48.189534 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c210a0c-2bd1-4910-83b3-38a06aed2a32 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:48.193224 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02adc17-28dc-405c-9f49-9cd693d517de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:48.193826 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02adc17-28dc-405c-9f49-9cd693d517de service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:48.194102 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02adc17-28dc-405c-9f49-9cd693d517de service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:48.194372 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02adc17-28dc-405c-9f49-9cd693d517de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:48.203321 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02adc17-28dc-405c-9f49-9cd693d517de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:48.204667 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02adc17-28dc-405c-9f49-9cd693d517de service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:48.214451 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1413/2819] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:17:48 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:17:48.662264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d8875-1e1f-474d-a5b1-fac91060b0ef None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:48.664135 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-6d1d8875-1e1f-474d-a5b1-fac91060b0ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:48.676670 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d8875-1e1f-474d-a5b1-fac91060b0ef None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:48.680241 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75d42a-dcaa-4214-bba8-7ea4d56f63a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:48.680882 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75d42a-dcaa-4214-bba8-7ea4d56f63a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:48.681158 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75d42a-dcaa-4214-bba8-7ea4d56f63a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:48.681427 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f75d42a-dcaa-4214-bba8-7ea4d56f63a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:48.746653 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75d42a-dcaa-4214-bba8-7ea4d56f63a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:48.748172 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f75d42a-dcaa-4214-bba8-7ea4d56f63a7 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:48.758854 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1407/2820] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:17:48 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:17:50.952095 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02adc17-28dc-405c-9f49-9cd693d517de service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:50.953582 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f02adc17-28dc-405c-9f49-9cd693d517de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:51.009742 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02adc17-28dc-405c-9f49-9cd693d517de service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:51.013340 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891dc663-e60f-4ef4-8aa8-3866377ab2ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:51.013933 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891dc663-e60f-4ef4-8aa8-3866377ab2ba service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:51.014211 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891dc663-e60f-4ef4-8aa8-3866377ab2ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:51.014483 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891dc663-e60f-4ef4-8aa8-3866377ab2ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:51.016283 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-891dc663-e60f-4ef4-8aa8-3866377ab2ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:51.094761 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891dc663-e60f-4ef4-8aa8-3866377ab2ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:51.098030 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891dc663-e60f-4ef4-8aa8-3866377ab2ba service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:51.110826 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1414/2821] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:17:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 10:17:54.759178 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75d42a-dcaa-4214-bba8-7ea4d56f63a7 service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:54.760633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3f75d42a-dcaa-4214-bba8-7ea4d56f63a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:54.775154 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f75d42a-dcaa-4214-bba8-7ea4d56f63a7 service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:54.779908 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7c42d-bfcb-495c-9af9-46cf76dce74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:54.781230 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7c42d-bfcb-495c-9af9-46cf76dce74d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:54.781230 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7c42d-bfcb-495c-9af9-46cf76dce74d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:54.781230 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7c42d-bfcb-495c-9af9-46cf76dce74d admin admin] PATH_INFO: `/v3/users/65963d36fc4741c69534b3dde1e00857` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:54.782193 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f7c42d-bfcb-495c-9af9-46cf76dce74d admin admin] RBAC: Authorizing `identity:delete_user(user_id=65963d36fc4741c69534b3dde1e00857)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:54.784175 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f7c42d-bfcb-495c-9af9-46cf76dce74d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:55.000706 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d4f7c42d-bfcb-495c-9af9-46cf76dce74d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65963d36fc4741c69534b3dde1e00857'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:55.060023 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d4f7c42d-bfcb-495c-9af9-46cf76dce74d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65963d36fc4741c69534b3dde1e00857'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:55.060666 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d4f7c42d-bfcb-495c-9af9-46cf76dce74d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65963d36fc4741c69534b3dde1e00857'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:55.061295 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d4f7c42d-bfcb-495c-9af9-46cf76dce74d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65963d36fc4741c69534b3dde1e00857'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:55.072450 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:17:55.074623 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d4f7c42d-bfcb-495c-9af9-46cf76dce74d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65963d36fc4741c69534b3dde1e00857'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:55.086408 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1408/2822] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 10:17:54 2022] DELETE /identity/v3/users/65963d36fc4741c69534b3dde1e00857 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:17:55.347289 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891dc663-e60f-4ef4-8aa8-3866377ab2ba service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:55.373959 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891dc663-e60f-4ef4-8aa8-3866377ab2ba service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:55.424174 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:17:55.432689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337bb316-0fa7-4808-bb5e-ac30cf7367fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:55.433319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bb316-0fa7-4808-bb5e-ac30cf7367fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:55.433600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bb316-0fa7-4808-bb5e-ac30cf7367fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:55.433876 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337bb316-0fa7-4808-bb5e-ac30cf7367fa admin admin] PATH_INFO: `/v3/projects/903eb64927ad43f1a2f1f2addb1c1b16` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:55.435087 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337bb316-0fa7-4808-bb5e-ac30cf7367fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=903eb64927ad43f1a2f1f2addb1c1b16)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:55.437026 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337bb316-0fa7-4808-bb5e-ac30cf7367fa admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:55.552262 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-337bb316-0fa7-4808-bb5e-ac30cf7367fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '903eb64927ad43f1a2f1f2addb1c1b16', 'request_id': 'req-337bb316-0fa7-4808-bb5e-ac30cf7367fa'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:55.620473 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-337bb316-0fa7-4808-bb5e-ac30cf7367fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '903eb64927ad43f1a2f1f2addb1c1b16', 'request_id': 'req-337bb316-0fa7-4808-bb5e-ac30cf7367fa'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:55.625989 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-337bb316-0fa7-4808-bb5e-ac30cf7367fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '903eb64927ad43f1a2f1f2addb1c1b16', 'request_id': 'req-337bb316-0fa7-4808-bb5e-ac30cf7367fa'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:55.637882 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-337bb316-0fa7-4808-bb5e-ac30cf7367fa admin admin] The token cache is being invalidate because project 903eb64927ad43f1a2f1f2addb1c1b16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 10:17:55.638164 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-337bb316-0fa7-4808-bb5e-ac30cf7367fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:55.638913 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-337bb316-0fa7-4808-bb5e-ac30cf7367fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:17:55.640386 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1415/2823] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 10:17:55 2022] DELETE /identity/v3/projects/903eb64927ad43f1a2f1f2addb1c1b16 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:17:55.645424 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba226fa-dc14-424c-8f87-4f5b804abc66 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:55.645708 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba226fa-dc14-424c-8f87-4f5b804abc66 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:55.645982 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba226fa-dc14-424c-8f87-4f5b804abc66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:55.665253 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-8ba226fa-dc14-424c-8f87-4f5b804abc66 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:17:55.666446 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8ba226fa-dc14-424c-8f87-4f5b804abc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:55.724430 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1409/2824] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:17:55.732251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b1b361-0efc-4e20-95f9-723e12bef784 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:55.732539 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b1b361-0efc-4e20-95f9-723e12bef784 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:55.732837 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b1b361-0efc-4e20-95f9-723e12bef784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:55.753190 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c3b1b361-0efc-4e20-95f9-723e12bef784 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:17:55.754375 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c3b1b361-0efc-4e20-95f9-723e12bef784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:55.762167 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1416/2825] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:17:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:17:55.766230 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba226fa-dc14-424c-8f87-4f5b804abc66 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:55.775878 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba226fa-dc14-424c-8f87-4f5b804abc66 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:55.780412 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d402ebef-2c99-4b03-bcb5-ef914055bd98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:55.781010 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402ebef-2c99-4b03-bcb5-ef914055bd98 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:55.781287 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402ebef-2c99-4b03-bcb5-ef914055bd98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:55.781558 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402ebef-2c99-4b03-bcb5-ef914055bd98 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:55.782207 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d402ebef-2c99-4b03-bcb5-ef914055bd98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 10:17:55.782508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d402ebef-2c99-4b03-bcb5-ef914055bd98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:55.784120 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d402ebef-2c99-4b03-bcb5-ef914055bd98 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:55.795384 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1410/2826] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 10:17:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 10:17:55.799830 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b1b361-0efc-4e20-95f9-723e12bef784 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:55.808856 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b1b361-0efc-4e20-95f9-723e12bef784 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:55.813368 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698d686-2b30-4df2-ba1c-4357403cfe6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:55.813955 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698d686-2b30-4df2-ba1c-4357403cfe6d admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:55.814234 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698d686-2b30-4df2-ba1c-4357403cfe6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:55.815102 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d698d686-2b30-4df2-ba1c-4357403cfe6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:55.815718 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d698d686-2b30-4df2-ba1c-4357403cfe6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:55.817586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d698d686-2b30-4df2-ba1c-4357403cfe6d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:55.919547 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1417/2827] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 10:17:55 2022] POST /identity/v3/projects => generated 367 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:17:55.924272 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d402ebef-2c99-4b03-bcb5-ef914055bd98 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:55.934323 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d402ebef-2c99-4b03-bcb5-ef914055bd98 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:55.989040 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49435f3-9730-41b8-aa82-32772316364d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:55.989567 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49435f3-9730-41b8-aa82-32772316364d admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:55.989848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49435f3-9730-41b8-aa82-32772316364d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:55.990123 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49435f3-9730-41b8-aa82-32772316364d admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:55.990784 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49435f3-9730-41b8-aa82-32772316364d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:55.992676 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49435f3-9730-41b8-aa82-32772316364d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:56.107163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1411/2828] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 10:17:55 2022] POST /identity/v3/users => generated 284 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:17:56.111988 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698d686-2b30-4df2-ba1c-4357403cfe6d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:56.122583 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d698d686-2b30-4df2-ba1c-4357403cfe6d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:56.127105 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79e3be-a5f8-4c8a-8ef7-7ac707e46c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:56.127695 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79e3be-a5f8-4c8a-8ef7-7ac707e46c83 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:56.127976 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79e3be-a5f8-4c8a-8ef7-7ac707e46c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:56.128274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79e3be-a5f8-4c8a-8ef7-7ac707e46c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:56.128935 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79e3be-a5f8-4c8a-8ef7-7ac707e46c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:56.130507 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db79e3be-a5f8-4c8a-8ef7-7ac707e46c83 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:56.140532 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1418/2829] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:17:56.148760 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49435f3-9730-41b8-aa82-32772316364d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:56.159443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49435f3-9730-41b8-aa82-32772316364d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:56.164102 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1acb96-45d2-45c5-84f9-3d4f6707af10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:56.164716 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1acb96-45d2-45c5-84f9-3d4f6707af10 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:56.164993 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1acb96-45d2-45c5-84f9-3d4f6707af10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:56.165350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1acb96-45d2-45c5-84f9-3d4f6707af10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:56.169093 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1acb96-45d2-45c5-84f9-3d4f6707af10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:56.169093 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1acb96-45d2-45c5-84f9-3d4f6707af10 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:56.178724 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1412/2830] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:17:56.182625 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79e3be-a5f8-4c8a-8ef7-7ac707e46c83 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:56.192607 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79e3be-a5f8-4c8a-8ef7-7ac707e46c83 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:56.197148 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a0945-c6f7-4e25-85b9-0a23a16bc4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:56.197785 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a0945-c6f7-4e25-85b9-0a23a16bc4e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:56.198064 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a0945-c6f7-4e25-85b9-0a23a16bc4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:56.198338 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a0945-c6f7-4e25-85b9-0a23a16bc4e7 admin admin] PATH_INFO: `/v3/projects/6450e66c96354bceb6c174df2b9bccc5/users/9a14ce541a204ad2b5925f313814ee75/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:56.214145 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245a0945-c6f7-4e25-85b9-0a23a16bc4e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6450e66c96354bceb6c174df2b9bccc5, user_id=9a14ce541a204ad2b5925f313814ee75, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:56.216778 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245a0945-c6f7-4e25-85b9-0a23a16bc4e7 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:56.283857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1419/2831] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 10:17:56 2022] PUT /identity/v3/projects/6450e66c96354bceb6c174df2b9bccc5/users/9a14ce541a204ad2b5925f313814ee75/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:17:56.295045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f5065-7164-4cad-899e-5dad96fb5b18 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:56.295328 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f5065-7164-4cad-899e-5dad96fb5b18 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:56.295598 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f5065-7164-4cad-899e-5dad96fb5b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:56.322222 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-dd0f5065-7164-4cad-899e-5dad96fb5b18 None None] MFA Rules not processed for user `9a14ce541a204ad2b5925f313814ee75`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:17:56.323452 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-dd0f5065-7164-4cad-899e-5dad96fb5b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:56.364004 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:17:56.390581 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1413/2832] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:17:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:17:56.398692 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a0945-c6f7-4e25-85b9-0a23a16bc4e7 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:17:56.400748 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-245a0945-c6f7-4e25-85b9-0a23a16bc4e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:17:56.458904 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a0945-c6f7-4e25-85b9-0a23a16bc4e7 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:17:56.462488 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb99f62-52dc-4acf-b55d-e008a23fcc4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:17:56.463112 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb99f62-52dc-4acf-b55d-e008a23fcc4b service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:17:56.463387 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb99f62-52dc-4acf-b55d-e008a23fcc4b service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:17:56.463656 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb99f62-52dc-4acf-b55d-e008a23fcc4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:17:56.474726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb99f62-52dc-4acf-b55d-e008a23fcc4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:17:56.476176 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb99f62-52dc-4acf-b55d-e008a23fcc4b service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:17:56.485405 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1420/2833] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:17:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:18:01.007050 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f5065-7164-4cad-899e-5dad96fb5b18 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:01.008480 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-dd0f5065-7164-4cad-899e-5dad96fb5b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:01.073064 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f5065-7164-4cad-899e-5dad96fb5b18 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:01.077555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682d82a-73fe-4816-8b8c-b088d5686407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:01.078178 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682d82a-73fe-4816-8b8c-b088d5686407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:01.079906 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682d82a-73fe-4816-8b8c-b088d5686407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:01.079906 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682d82a-73fe-4816-8b8c-b088d5686407 admin admin] PATH_INFO: `/v3/users/319cda3819914949ae46ff5d38303105` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:01.079906 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0682d82a-73fe-4816-8b8c-b088d5686407 admin admin] RBAC: Authorizing `identity:delete_user(user_id=319cda3819914949ae46ff5d38303105)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:01.082143 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0682d82a-73fe-4816-8b8c-b088d5686407 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:01.342801 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0682d82a-73fe-4816-8b8c-b088d5686407 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '319cda3819914949ae46ff5d38303105'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:01.397749 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0682d82a-73fe-4816-8b8c-b088d5686407 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '319cda3819914949ae46ff5d38303105'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:01.398354 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0682d82a-73fe-4816-8b8c-b088d5686407 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '319cda3819914949ae46ff5d38303105'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:01.398986 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0682d82a-73fe-4816-8b8c-b088d5686407 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '319cda3819914949ae46ff5d38303105'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:01.411503 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-0682d82a-73fe-4816-8b8c-b088d5686407 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '319cda3819914949ae46ff5d38303105'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:01.423498 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1414/2834] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 10:18:01 2022] DELETE /identity/v3/users/319cda3819914949ae46ff5d38303105 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:18:01.428690 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8ccc6-4465-496c-824a-2213b3cee982 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:01.429160 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8ccc6-4465-496c-824a-2213b3cee982 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:01.429256 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8ccc6-4465-496c-824a-2213b3cee982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:01.447668 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-97b8ccc6-4465-496c-824a-2213b3cee982 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:18:01.448861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-97b8ccc6-4465-496c-824a-2213b3cee982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:01.465388 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:01.507003 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1421/2835] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:18:01.516355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682d82a-73fe-4816-8b8c-b088d5686407 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:01.517841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0682d82a-73fe-4816-8b8c-b088d5686407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:01.575530 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682d82a-73fe-4816-8b8c-b088d5686407 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:01.579183 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9d73c-be4c-4cbe-a350-f9877884c1cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:01.579792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9d73c-be4c-4cbe-a350-f9877884c1cd service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:01.580071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9d73c-be4c-4cbe-a350-f9877884c1cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:01.580340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9d73c-be4c-4cbe-a350-f9877884c1cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:01.589845 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9d73c-be4c-4cbe-a350-f9877884c1cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:01.591296 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9d73c-be4c-4cbe-a350-f9877884c1cd service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:01.601766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1415/2836] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:18:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:18:02.062678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8ccc6-4465-496c-824a-2213b3cee982 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:02.064376 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-97b8ccc6-4465-496c-824a-2213b3cee982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:02.080466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8ccc6-4465-496c-824a-2213b3cee982 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:02.085833 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd04414-fbfa-4035-a9e6-1fe1cd88279a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:02.088010 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd04414-fbfa-4035-a9e6-1fe1cd88279a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:02.088010 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd04414-fbfa-4035-a9e6-1fe1cd88279a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:02.088010 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd04414-fbfa-4035-a9e6-1fe1cd88279a admin admin] PATH_INFO: `/v3/projects/bbfbe0266fba47eaa732281a886df41d` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:02.088429 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd04414-fbfa-4035-a9e6-1fe1cd88279a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbfbe0266fba47eaa732281a886df41d)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:02.090503 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd04414-fbfa-4035-a9e6-1fe1cd88279a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:02.200989 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4fd04414-fbfa-4035-a9e6-1fe1cd88279a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbfbe0266fba47eaa732281a886df41d', 'request_id': 'req-4fd04414-fbfa-4035-a9e6-1fe1cd88279a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:02.266655 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4fd04414-fbfa-4035-a9e6-1fe1cd88279a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbfbe0266fba47eaa732281a886df41d', 'request_id': 'req-4fd04414-fbfa-4035-a9e6-1fe1cd88279a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:02.274155 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4fd04414-fbfa-4035-a9e6-1fe1cd88279a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbfbe0266fba47eaa732281a886df41d', 'request_id': 'req-4fd04414-fbfa-4035-a9e6-1fe1cd88279a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:02.288417 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4fd04414-fbfa-4035-a9e6-1fe1cd88279a admin admin] The token cache is being invalidate because project bbfbe0266fba47eaa732281a886df41d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 10:18:02.288714 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4fd04414-fbfa-4035-a9e6-1fe1cd88279a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:02.289377 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4fd04414-fbfa-4035-a9e6-1fe1cd88279a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:02.291193 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1422/2837] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 10:18:02 2022] DELETE /identity/v3/projects/bbfbe0266fba47eaa732281a886df41d => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:18:02.295224 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9d73c-be4c-4cbe-a350-f9877884c1cd service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:02.300206 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:02.305652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9d73c-be4c-4cbe-a350-f9877884c1cd service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:02.365045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b52aa-d645-4c01-ba45-47ae05cc8d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:02.365615 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b52aa-d645-4c01-ba45-47ae05cc8d17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:02.365918 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b52aa-d645-4c01-ba45-47ae05cc8d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:02.366199 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b52aa-d645-4c01-ba45-47ae05cc8d17 admin admin] PATH_INFO: `/v3/users/95226d62d81140189cbd84fa3555bc87` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:02.367409 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b52aa-d645-4c01-ba45-47ae05cc8d17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95226d62d81140189cbd84fa3555bc87)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:02.369379 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b52aa-d645-4c01-ba45-47ae05cc8d17 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:02.618700 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-277b52aa-d645-4c01-ba45-47ae05cc8d17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95226d62d81140189cbd84fa3555bc87'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:02.695212 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-277b52aa-d645-4c01-ba45-47ae05cc8d17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95226d62d81140189cbd84fa3555bc87'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:02.696009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-277b52aa-d645-4c01-ba45-47ae05cc8d17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95226d62d81140189cbd84fa3555bc87'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:02.696738 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-277b52aa-d645-4c01-ba45-47ae05cc8d17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95226d62d81140189cbd84fa3555bc87'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:02.709766 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-277b52aa-d645-4c01-ba45-47ae05cc8d17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95226d62d81140189cbd84fa3555bc87'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:02.725047 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1416/2838] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 10:18:02 2022] DELETE /identity/v3/users/95226d62d81140189cbd84fa3555bc87 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:18:03.102469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd04414-fbfa-4035-a9e6-1fe1cd88279a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:03.103988 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4fd04414-fbfa-4035-a9e6-1fe1cd88279a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:03.118047 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd04414-fbfa-4035-a9e6-1fe1cd88279a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:03.176197 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee5a14b-b059-4e2e-beee-66ec80eb5322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:03.176755 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5a14b-b059-4e2e-beee-66ec80eb5322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:03.177030 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5a14b-b059-4e2e-beee-66ec80eb5322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:03.177300 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5a14b-b059-4e2e-beee-66ec80eb5322 admin admin] PATH_INFO: `/v3/projects/afc76ac80d5046a3a31be97a59197aa7` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:03.178405 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee5a14b-b059-4e2e-beee-66ec80eb5322 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afc76ac80d5046a3a31be97a59197aa7)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:03.180326 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee5a14b-b059-4e2e-beee-66ec80eb5322 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:03.273602 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:03.274917 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dee5a14b-b059-4e2e-beee-66ec80eb5322 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afc76ac80d5046a3a31be97a59197aa7', 'request_id': 'req-dee5a14b-b059-4e2e-beee-66ec80eb5322'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:03.404180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dee5a14b-b059-4e2e-beee-66ec80eb5322 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afc76ac80d5046a3a31be97a59197aa7', 'request_id': 'req-dee5a14b-b059-4e2e-beee-66ec80eb5322'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:03.410836 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dee5a14b-b059-4e2e-beee-66ec80eb5322 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afc76ac80d5046a3a31be97a59197aa7', 'request_id': 'req-dee5a14b-b059-4e2e-beee-66ec80eb5322'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:03.421670 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dee5a14b-b059-4e2e-beee-66ec80eb5322 admin admin] The token cache is being invalidate because project afc76ac80d5046a3a31be97a59197aa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 10:18:03.427377 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dee5a14b-b059-4e2e-beee-66ec80eb5322 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:03.427377 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dee5a14b-b059-4e2e-beee-66ec80eb5322 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:03.427377 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1423/2839] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 10:18:03 2022] DELETE /identity/v3/projects/afc76ac80d5046a3a31be97a59197aa7 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:18:03.429514 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db89ab4-dd5f-4cf2-8260-be7b15825269 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:03.429968 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db89ab4-dd5f-4cf2-8260-be7b15825269 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:03.430057 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db89ab4-dd5f-4cf2-8260-be7b15825269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:03.449526 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0db89ab4-dd5f-4cf2-8260-be7b15825269 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:18:03.450792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0db89ab4-dd5f-4cf2-8260-be7b15825269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:03.508922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1417/2840] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:18:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:18:03.516786 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c1e37-84df-4d37-829a-d746dd756701 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:03.517076 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c1e37-84df-4d37-829a-d746dd756701 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:03.517349 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c1e37-84df-4d37-829a-d746dd756701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:03.536983 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6d0c1e37-84df-4d37-829a-d746dd756701 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:18:03.538188 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6d0c1e37-84df-4d37-829a-d746dd756701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:03.545930 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1424/2841] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:18:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:18:03.550651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db89ab4-dd5f-4cf2-8260-be7b15825269 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:03.560208 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db89ab4-dd5f-4cf2-8260-be7b15825269 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:03.564734 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64699ebc-3a6d-46e8-b197-18b867664d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:03.565333 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64699ebc-3a6d-46e8-b197-18b867664d24 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:03.565614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64699ebc-3a6d-46e8-b197-18b867664d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:03.566097 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64699ebc-3a6d-46e8-b197-18b867664d24 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:03.566792 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64699ebc-3a6d-46e8-b197-18b867664d24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 10:18:03.567100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64699ebc-3a6d-46e8-b197-18b867664d24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:03.568712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64699ebc-3a6d-46e8-b197-18b867664d24 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:03.580173 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1418/2842] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 10:18:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 10:18:03.584449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c1e37-84df-4d37-829a-d746dd756701 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:03.593787 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c1e37-84df-4d37-829a-d746dd756701 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:03.599201 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb38f7b-fb4b-4910-b913-3e7a4df56822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:03.599791 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb38f7b-fb4b-4910-b913-3e7a4df56822 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:03.600106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb38f7b-fb4b-4910-b913-3e7a4df56822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:03.600369 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb38f7b-fb4b-4910-b913-3e7a4df56822 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:03.601001 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb38f7b-fb4b-4910-b913-3e7a4df56822 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:03.602911 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb38f7b-fb4b-4910-b913-3e7a4df56822 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:03.718775 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1425/2843] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 10:18:03 2022] POST /identity/v3/projects => generated 379 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:18:03.723786 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64699ebc-3a6d-46e8-b197-18b867664d24 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:03.733726 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64699ebc-3a6d-46e8-b197-18b867664d24 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:03.741136 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:03.792961 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c51dd-1fcd-4abb-bbac-be959da6d0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:03.796521 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c51dd-1fcd-4abb-bbac-be959da6d0ec admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:03.796815 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c51dd-1fcd-4abb-bbac-be959da6d0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:03.797100 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c51dd-1fcd-4abb-bbac-be959da6d0ec admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:03.797753 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7c51dd-1fcd-4abb-bbac-be959da6d0ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:03.799717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7c51dd-1fcd-4abb-bbac-be959da6d0ec admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:03.920902 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1419/2844] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 10:18:03 2022] POST /identity/v3/users => generated 290 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:18:03.922340 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb38f7b-fb4b-4910-b913-3e7a4df56822 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:03.932576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb38f7b-fb4b-4910-b913-3e7a4df56822 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:03.937085 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048240f-331e-4658-9912-0c55531d2846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:03.937676 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048240f-331e-4658-9912-0c55531d2846 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:03.937957 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048240f-331e-4658-9912-0c55531d2846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:03.938229 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8048240f-331e-4658-9912-0c55531d2846 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:03.938907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8048240f-331e-4658-9912-0c55531d2846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:03.940559 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8048240f-331e-4658-9912-0c55531d2846 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:03.950234 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1426/2845] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:18:03.954440 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c51dd-1fcd-4abb-bbac-be959da6d0ec admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:03.965175 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c51dd-1fcd-4abb-bbac-be959da6d0ec admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:03.969885 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a244d-fc61-4d40-aa4b-ee85b98c2487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:03.970423 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a244d-fc61-4d40-aa4b-ee85b98c2487 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:03.970733 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a244d-fc61-4d40-aa4b-ee85b98c2487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:03.971005 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a244d-fc61-4d40-aa4b-ee85b98c2487 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:03.971652 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17a244d-fc61-4d40-aa4b-ee85b98c2487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:03.974148 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17a244d-fc61-4d40-aa4b-ee85b98c2487 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:03.984561 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1420/2846] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:18:03.988687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048240f-331e-4658-9912-0c55531d2846 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:03.998420 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8048240f-331e-4658-9912-0c55531d2846 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:04.003153 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee0ffb-c9ad-4c3d-9d7d-5215beaf0371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:04.003787 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee0ffb-c9ad-4c3d-9d7d-5215beaf0371 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:04.004066 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee0ffb-c9ad-4c3d-9d7d-5215beaf0371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:04.004340 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ee0ffb-c9ad-4c3d-9d7d-5215beaf0371 admin admin] PATH_INFO: `/v3/projects/068958eb674446f1923d4e79d782cfab/users/f78a8d2ab1fc4e8abd13a7ec3bba33ea/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:04.020818 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ee0ffb-c9ad-4c3d-9d7d-5215beaf0371 admin admin] RBAC: Authorizing `identity:create_grant(project_id=068958eb674446f1923d4e79d782cfab, user_id=f78a8d2ab1fc4e8abd13a7ec3bba33ea, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:04.023485 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ee0ffb-c9ad-4c3d-9d7d-5215beaf0371 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:04.118401 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1427/2847] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 10:18:03 2022] PUT /identity/v3/projects/068958eb674446f1923d4e79d782cfab/users/f78a8d2ab1fc4e8abd13a7ec3bba33ea/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:18:04.130259 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4301255-d25a-4773-accf-ba321dec6c68 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:04.130614 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4301255-d25a-4773-accf-ba321dec6c68 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:04.130929 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4301255-d25a-4773-accf-ba321dec6c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:04.159532 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f4301255-d25a-4773-accf-ba321dec6c68 None None] MFA Rules not processed for user `f78a8d2ab1fc4e8abd13a7ec3bba33ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:18:04.160765 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f4301255-d25a-4773-accf-ba321dec6c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:04.224229 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1421/2848] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:18:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:18:04.228650 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee0ffb-c9ad-4c3d-9d7d-5215beaf0371 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:04.290797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ee0ffb-c9ad-4c3d-9d7d-5215beaf0371 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:04.295584 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3631ca6-e6e5-4c3b-bb54-f21aafc99ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:04.296186 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3631ca6-e6e5-4c3b-bb54-f21aafc99ba2 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:04.296466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3631ca6-e6e5-4c3b-bb54-f21aafc99ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:04.296741 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3631ca6-e6e5-4c3b-bb54-f21aafc99ba2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:04.297383 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3631ca6-e6e5-4c3b-bb54-f21aafc99ba2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:04.299354 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3631ca6-e6e5-4c3b-bb54-f21aafc99ba2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:04.416118 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1428/2849] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 10:18:04 2022] POST /identity/v3/projects => generated 377 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:18:04.420172 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4301255-d25a-4773-accf-ba321dec6c68 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:04.431486 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4301255-d25a-4773-accf-ba321dec6c68 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:04.489137 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05a7d0-a59f-4881-b6c6-a05d278f4777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:04.489662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05a7d0-a59f-4881-b6c6-a05d278f4777 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:04.489948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05a7d0-a59f-4881-b6c6-a05d278f4777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:04.490280 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05a7d0-a59f-4881-b6c6-a05d278f4777 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:04.490958 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d05a7d0-a59f-4881-b6c6-a05d278f4777 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:04.492942 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d05a7d0-a59f-4881-b6c6-a05d278f4777 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:04.593739 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1422/2850] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 10:18:04 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:18:04.600175 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3631ca6-e6e5-4c3b-bb54-f21aafc99ba2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:04.610517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3631ca6-e6e5-4c3b-bb54-f21aafc99ba2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:04.615751 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbd978-c0a8-4099-b6ac-b4852f7e619b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:04.616348 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbd978-c0a8-4099-b6ac-b4852f7e619b admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:04.616624 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbd978-c0a8-4099-b6ac-b4852f7e619b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:04.616893 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbd978-c0a8-4099-b6ac-b4852f7e619b admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:04.617526 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bbd978-c0a8-4099-b6ac-b4852f7e619b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:04.619130 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bbd978-c0a8-4099-b6ac-b4852f7e619b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:04.630349 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1429/2851] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:18:04.634409 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05a7d0-a59f-4881-b6c6-a05d278f4777 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:04.645311 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05a7d0-a59f-4881-b6c6-a05d278f4777 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:04.650382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464b4fd-e4fa-4455-916a-3dfa65129723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:04.651028 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464b4fd-e4fa-4455-916a-3dfa65129723 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:04.651312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464b4fd-e4fa-4455-916a-3dfa65129723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:04.651583 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464b4fd-e4fa-4455-916a-3dfa65129723 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:04.652239 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464b4fd-e4fa-4455-916a-3dfa65129723 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:04.653857 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464b4fd-e4fa-4455-916a-3dfa65129723 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:04.655441 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:04.666621 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1423/2852] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:18:04.671483 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbd978-c0a8-4099-b6ac-b4852f7e619b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:04.681980 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbd978-c0a8-4099-b6ac-b4852f7e619b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:04.686506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba2d37-ab41-4140-9643-c90ec7a53349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:04.687161 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba2d37-ab41-4140-9643-c90ec7a53349 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:04.687440 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba2d37-ab41-4140-9643-c90ec7a53349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:04.687708 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ba2d37-ab41-4140-9643-c90ec7a53349 admin admin] PATH_INFO: `/v3/projects/f41838411bab42e7981f4f1f73104cc2/users/f66a019577c54cababecaa155e04e82b/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:04.703573 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ba2d37-ab41-4140-9643-c90ec7a53349 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f41838411bab42e7981f4f1f73104cc2, user_id=f66a019577c54cababecaa155e04e82b, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:04.706213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ba2d37-ab41-4140-9643-c90ec7a53349 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:04.769004 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1430/2853] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 10:18:04 2022] PUT /identity/v3/projects/f41838411bab42e7981f4f1f73104cc2/users/f66a019577c54cababecaa155e04e82b/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:18:04.779228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4be7fa-4c62-4a76-8866-cbf387981b28 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:04.779664 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4be7fa-4c62-4a76-8866-cbf387981b28 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:04.780026 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4be7fa-4c62-4a76-8866-cbf387981b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:04.811650 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-0c4be7fa-4c62-4a76-8866-cbf387981b28 None None] MFA Rules not processed for user `f66a019577c54cababecaa155e04e82b`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:18:04.812868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0c4be7fa-4c62-4a76-8866-cbf387981b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:04.876563 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1424/2854] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:18:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:18:04.881768 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba2d37-ab41-4140-9643-c90ec7a53349 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:04.906543 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:04.942814 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ba2d37-ab41-4140-9643-c90ec7a53349 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:04.947854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8abb045-f41c-4734-a33f-337c5d7a9a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:04.948449 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abb045-f41c-4734-a33f-337c5d7a9a35 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:04.948727 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abb045-f41c-4734-a33f-337c5d7a9a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:04.948992 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8abb045-f41c-4734-a33f-337c5d7a9a35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:04.949619 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8abb045-f41c-4734-a33f-337c5d7a9a35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:04.951512 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8abb045-f41c-4734-a33f-337c5d7a9a35 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:05.029814 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1431/2855] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 10:18:04 2022] POST /identity/v3/projects => generated 377 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:18:05.033841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4be7fa-4c62-4a76-8866-cbf387981b28 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:05.044520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4be7fa-4c62-4a76-8866-cbf387981b28 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:05.101704 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be197d4-7515-4bfc-beb5-90326caf0d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:05.105755 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be197d4-7515-4bfc-beb5-90326caf0d8f admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:05.105755 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be197d4-7515-4bfc-beb5-90326caf0d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:05.105755 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be197d4-7515-4bfc-beb5-90326caf0d8f admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:05.105755 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be197d4-7515-4bfc-beb5-90326caf0d8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:05.105755 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be197d4-7515-4bfc-beb5-90326caf0d8f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:05.203662 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1425/2856] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 10:18:05 2022] POST /identity/v3/users => generated 295 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:18:05.207762 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8abb045-f41c-4734-a33f-337c5d7a9a35 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:05.218756 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8abb045-f41c-4734-a33f-337c5d7a9a35 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:05.223707 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d601cd-fc28-4171-9f40-30457ec05cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:05.224305 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d601cd-fc28-4171-9f40-30457ec05cea admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:05.224588 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d601cd-fc28-4171-9f40-30457ec05cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:05.224854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d601cd-fc28-4171-9f40-30457ec05cea admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:05.225491 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d601cd-fc28-4171-9f40-30457ec05cea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:05.227112 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d601cd-fc28-4171-9f40-30457ec05cea admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:05.238065 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1432/2857] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:18:05.242312 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be197d4-7515-4bfc-beb5-90326caf0d8f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:05.252181 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be197d4-7515-4bfc-beb5-90326caf0d8f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:05.257456 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea388d29-c450-4752-8403-1482b533a010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:05.258106 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea388d29-c450-4752-8403-1482b533a010 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:05.258429 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea388d29-c450-4752-8403-1482b533a010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:05.258730 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea388d29-c450-4752-8403-1482b533a010 admin admin] PATH_INFO: `/v3/projects/f0fd972f534545eb86cac5c6aad18bc3/users/f3d74697fa854f7a962af2b19c7b7ba0/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:05.273243 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea388d29-c450-4752-8403-1482b533a010 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0fd972f534545eb86cac5c6aad18bc3, user_id=f3d74697fa854f7a962af2b19c7b7ba0, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:05.275981 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea388d29-c450-4752-8403-1482b533a010 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:05.341292 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1426/2858] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 10:18:05 2022] PUT /identity/v3/projects/f0fd972f534545eb86cac5c6aad18bc3/users/f3d74697fa854f7a962af2b19c7b7ba0/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:18:05.348229 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d601cd-fc28-4171-9f40-30457ec05cea admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:05.358537 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d601cd-fc28-4171-9f40-30457ec05cea admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:05.413334 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a596d86-64f0-4d06-8dc9-202cf97d87db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:05.413944 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a596d86-64f0-4d06-8dc9-202cf97d87db admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:05.414251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a596d86-64f0-4d06-8dc9-202cf97d87db admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:05.414524 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a596d86-64f0-4d06-8dc9-202cf97d87db admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:05.415189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a596d86-64f0-4d06-8dc9-202cf97d87db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:05.416750 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a596d86-64f0-4d06-8dc9-202cf97d87db admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:05.426683 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1433/2859] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:18:05.430910 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea388d29-c450-4752-8403-1482b533a010 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:05.441739 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea388d29-c450-4752-8403-1482b533a010 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:05.446291 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee4152-8e8a-47f8-ac76-f8a7fdcee448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:05.446963 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee4152-8e8a-47f8-ac76-f8a7fdcee448 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:05.447245 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee4152-8e8a-47f8-ac76-f8a7fdcee448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:05.447518 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee4152-8e8a-47f8-ac76-f8a7fdcee448 admin admin] PATH_INFO: `/v3/projects/f0fd972f534545eb86cac5c6aad18bc3/users/f3d74697fa854f7a962af2b19c7b7ba0/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:05.449414 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee4152-8e8a-47f8-ac76-f8a7fdcee448 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0fd972f534545eb86cac5c6aad18bc3, user_id=f3d74697fa854f7a962af2b19c7b7ba0, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:05.452045 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee4152-8e8a-47f8-ac76-f8a7fdcee448 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:05.462186 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1427/2860] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 10:18:05 2022] PUT /identity/v3/projects/f0fd972f534545eb86cac5c6aad18bc3/users/f3d74697fa854f7a962af2b19c7b7ba0/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:18:05.472183 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d454a4-57c8-4208-bc16-69b28f9e9082 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:05.472492 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d454a4-57c8-4208-bc16-69b28f9e9082 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:05.472772 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d454a4-57c8-4208-bc16-69b28f9e9082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:05.500323 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-11d454a4-57c8-4208-bc16-69b28f9e9082 None None] MFA Rules not processed for user `f3d74697fa854f7a962af2b19c7b7ba0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:18:05.501525 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-11d454a4-57c8-4208-bc16-69b28f9e9082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:05.569421 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1434/2861] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:18:05 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 10:18:05.581055 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee4152-8e8a-47f8-ac76-f8a7fdcee448 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:05.583349 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b3ee4152-8e8a-47f8-ac76-f8a7fdcee448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:05.625791 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:05.641713 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee4152-8e8a-47f8-ac76-f8a7fdcee448 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:05.647628 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd335ac-dc7f-4daf-8b02-0e7bf02ce88b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:05.647628 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd335ac-dc7f-4daf-8b02-0e7bf02ce88b service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:05.647628 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd335ac-dc7f-4daf-8b02-0e7bf02ce88b service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:05.647628 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd335ac-dc7f-4daf-8b02-0e7bf02ce88b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:05.656948 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd335ac-dc7f-4daf-8b02-0e7bf02ce88b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:05.658407 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd335ac-dc7f-4daf-8b02-0e7bf02ce88b service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:05.667811 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1428/2862] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:18:05 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:18:05.773041 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d454a4-57c8-4208-bc16-69b28f9e9082 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:05.774534 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-11d454a4-57c8-4208-bc16-69b28f9e9082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:05.787060 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d454a4-57c8-4208-bc16-69b28f9e9082 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:05.790555 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0db125-c5b2-4eb7-9088-31c834ad8203 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:05.791180 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0db125-c5b2-4eb7-9088-31c834ad8203 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:05.791460 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0db125-c5b2-4eb7-9088-31c834ad8203 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:05.791731 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0db125-c5b2-4eb7-9088-31c834ad8203 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:05.804900 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:05.848359 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0db125-c5b2-4eb7-9088-31c834ad8203 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:05.849788 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0db125-c5b2-4eb7-9088-31c834ad8203 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:05.860455 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1435/2863] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:18:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:18:06.379995 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd335ac-dc7f-4daf-8b02-0e7bf02ce88b service cinder] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:06.381429 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-dcd335ac-dc7f-4daf-8b02-0e7bf02ce88b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:06.445922 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd335ac-dc7f-4daf-8b02-0e7bf02ce88b service cinder] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:06.450648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb8748-0368-4a89-ab55-ce98f1b42c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:06.451309 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb8748-0368-4a89-ab55-ce98f1b42c52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:06.451593 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb8748-0368-4a89-ab55-ce98f1b42c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:06.451865 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cb8748-0368-4a89-ab55-ce98f1b42c52 admin admin] PATH_INFO: `/v3/users/9a14ce541a204ad2b5925f313814ee75` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:06.452983 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb8748-0368-4a89-ab55-ce98f1b42c52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a14ce541a204ad2b5925f313814ee75)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:06.454911 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cb8748-0368-4a89-ab55-ce98f1b42c52 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:06.658326 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-72cb8748-0368-4a89-ab55-ce98f1b42c52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a14ce541a204ad2b5925f313814ee75'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:06.723491 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-72cb8748-0368-4a89-ab55-ce98f1b42c52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a14ce541a204ad2b5925f313814ee75'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:06.724111 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-72cb8748-0368-4a89-ab55-ce98f1b42c52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a14ce541a204ad2b5925f313814ee75'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:06.724722 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-72cb8748-0368-4a89-ab55-ce98f1b42c52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a14ce541a204ad2b5925f313814ee75'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:06.739552 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-72cb8748-0368-4a89-ab55-ce98f1b42c52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a14ce541a204ad2b5925f313814ee75'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:06.752287 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1429/2864] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 10:18:06 2022] DELETE /identity/v3/users/9a14ce541a204ad2b5925f313814ee75 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:18:06.757264 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108dcccc-96dd-4cca-b183-9986052fa1bd None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:06.757787 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108dcccc-96dd-4cca-b183-9986052fa1bd None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:06.757787 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108dcccc-96dd-4cca-b183-9986052fa1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:06.776676 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-108dcccc-96dd-4cca-b183-9986052fa1bd None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:18:06.777929 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-108dcccc-96dd-4cca-b183-9986052fa1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:06.835324 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1436/2865] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:18:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:18:06.845295 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb8748-0368-4a89-ab55-ce98f1b42c52 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:06.846826 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-72cb8748-0368-4a89-ab55-ce98f1b42c52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:06.905760 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cb8748-0368-4a89-ab55-ce98f1b42c52 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:06.909386 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05612db3-c91b-414d-a611-31d17ba31b01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:06.909993 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05612db3-c91b-414d-a611-31d17ba31b01 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:06.910272 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05612db3-c91b-414d-a611-31d17ba31b01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:06.910546 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05612db3-c91b-414d-a611-31d17ba31b01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:06.919413 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05612db3-c91b-414d-a611-31d17ba31b01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:06.922476 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05612db3-c91b-414d-a611-31d17ba31b01 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:06.930680 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1430/2866] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:18:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:18:07.308427 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108dcccc-96dd-4cca-b183-9986052fa1bd None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:07.309845 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-108dcccc-96dd-4cca-b183-9986052fa1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:07.321548 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108dcccc-96dd-4cca-b183-9986052fa1bd None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:07.327814 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8751f2-989a-4a9a-a343-bec695e0d295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:07.327814 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8751f2-989a-4a9a-a343-bec695e0d295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:07.327814 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8751f2-989a-4a9a-a343-bec695e0d295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:07.327814 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8751f2-989a-4a9a-a343-bec695e0d295 admin admin] PATH_INFO: `/v3/projects/6450e66c96354bceb6c174df2b9bccc5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:07.328444 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8751f2-989a-4a9a-a343-bec695e0d295 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6450e66c96354bceb6c174df2b9bccc5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:07.330326 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8751f2-989a-4a9a-a343-bec695e0d295 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:07.449281 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ae8751f2-989a-4a9a-a343-bec695e0d295 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6450e66c96354bceb6c174df2b9bccc5', 'request_id': 'req-ae8751f2-989a-4a9a-a343-bec695e0d295'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:07.539886 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ae8751f2-989a-4a9a-a343-bec695e0d295 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6450e66c96354bceb6c174df2b9bccc5', 'request_id': 'req-ae8751f2-989a-4a9a-a343-bec695e0d295'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:07.545554 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ae8751f2-989a-4a9a-a343-bec695e0d295 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6450e66c96354bceb6c174df2b9bccc5', 'request_id': 'req-ae8751f2-989a-4a9a-a343-bec695e0d295'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:07.556881 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ae8751f2-989a-4a9a-a343-bec695e0d295 admin admin] The token cache is being invalidate because project 6450e66c96354bceb6c174df2b9bccc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 10:18:07.557199 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ae8751f2-989a-4a9a-a343-bec695e0d295 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:07.557837 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ae8751f2-989a-4a9a-a343-bec695e0d295 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:07.559380 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1437/2867] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 10:18:07 2022] DELETE /identity/v3/projects/6450e66c96354bceb6c174df2b9bccc5 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:18:07.564705 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfed8e-2c9f-4f0f-85e4-89a0f430de1d None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:07.565172 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfed8e-2c9f-4f0f-85e4-89a0f430de1d None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:07.565260 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdfed8e-2c9f-4f0f-85e4-89a0f430de1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:07.573236 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:07.586028 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ccdfed8e-2c9f-4f0f-85e4-89a0f430de1d None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:18:07.587246 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ccdfed8e-2c9f-4f0f-85e4-89a0f430de1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:07.661477 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1431/2868] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:18:07.661637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e32486-9949-432a-b5e5-e8e25bce4a9b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:07.661637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e32486-9949-432a-b5e5-e8e25bce4a9b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:07.661637 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e32486-9949-432a-b5e5-e8e25bce4a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:07.679930 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-37e32486-9949-432a-b5e5-e8e25bce4a9b None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:18:07.681140 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-37e32486-9949-432a-b5e5-e8e25bce4a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:07.689063 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1438/2869] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:18:07.693467 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdfed8e-2c9f-4f0f-85e4-89a0f430de1d None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:07.703238 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdfed8e-2c9f-4f0f-85e4-89a0f430de1d None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:07.707800 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa8b26-15ca-45eb-9899-f7210c4d73b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:07.708881 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa8b26-15ca-45eb-9899-f7210c4d73b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:07.709168 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa8b26-15ca-45eb-9899-f7210c4d73b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:07.709440 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa8b26-15ca-45eb-9899-f7210c4d73b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:07.710086 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa8b26-15ca-45eb-9899-f7210c4d73b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 10:18:07.710391 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa8b26-15ca-45eb-9899-f7210c4d73b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:07.712000 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa8b26-15ca-45eb-9899-f7210c4d73b4 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:07.723786 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1432/2870] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 10:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 10:18:07.728684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e32486-9949-432a-b5e5-e8e25bce4a9b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:07.737944 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e32486-9949-432a-b5e5-e8e25bce4a9b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:07.743129 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adeeca7-8dcf-45d2-ad92-b73167ab6a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:07.743792 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adeeca7-8dcf-45d2-ad92-b73167ab6a9b admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:07.744183 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adeeca7-8dcf-45d2-ad92-b73167ab6a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:07.744565 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adeeca7-8dcf-45d2-ad92-b73167ab6a9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:07.744957 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adeeca7-8dcf-45d2-ad92-b73167ab6a9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:07.747121 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adeeca7-8dcf-45d2-ad92-b73167ab6a9b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:07.867857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1439/2871] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 10:18:07 2022] POST /identity/v3/projects => generated 373 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 10:18:07.879574 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa8b26-15ca-45eb-9899-f7210c4d73b4 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:07.898351 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa8b26-15ca-45eb-9899-f7210c4d73b4 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:07.956263 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020362f6-003a-44d2-99df-2eeee4f7eeab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:07.956799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020362f6-003a-44d2-99df-2eeee4f7eeab admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:07.957079 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020362f6-003a-44d2-99df-2eeee4f7eeab admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:07.957350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020362f6-003a-44d2-99df-2eeee4f7eeab admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:07.957984 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020362f6-003a-44d2-99df-2eeee4f7eeab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:07.960533 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020362f6-003a-44d2-99df-2eeee4f7eeab admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:08.079856 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1433/2872] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 10:18:07 2022] POST /identity/v3/users => generated 287 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 10:18:08.086754 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adeeca7-8dcf-45d2-ad92-b73167ab6a9b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:08.098320 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adeeca7-8dcf-45d2-ad92-b73167ab6a9b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:08.103206 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c99a1-9ac1-4784-bb02-372035f09821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:08.103882 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c99a1-9ac1-4784-bb02-372035f09821 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:08.104231 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c99a1-9ac1-4784-bb02-372035f09821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:08.104571 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c99a1-9ac1-4784-bb02-372035f09821 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:08.105301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c99a1-9ac1-4784-bb02-372035f09821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:08.106971 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c99a1-9ac1-4784-bb02-372035f09821 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:08.112684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:08.119328 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1440/2873] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:18:08.132404 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020362f6-003a-44d2-99df-2eeee4f7eeab admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:08.142381 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020362f6-003a-44d2-99df-2eeee4f7eeab admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:08.147200 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf291dd-2cde-43eb-b9a0-ec061c14fab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:08.147808 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf291dd-2cde-43eb-b9a0-ec061c14fab5 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:08.148091 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf291dd-2cde-43eb-b9a0-ec061c14fab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:08.148365 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf291dd-2cde-43eb-b9a0-ec061c14fab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:08.149009 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf291dd-2cde-43eb-b9a0-ec061c14fab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:08.150581 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf291dd-2cde-43eb-b9a0-ec061c14fab5 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:08.161131 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1434/2874] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:18:08.166245 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c99a1-9ac1-4784-bb02-372035f09821 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:08.176353 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c99a1-9ac1-4784-bb02-372035f09821 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:08.181434 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b699ede-e7d7-462c-8fa4-cab8919da8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:08.182085 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b699ede-e7d7-462c-8fa4-cab8919da8fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:08.182368 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b699ede-e7d7-462c-8fa4-cab8919da8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:08.182647 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b699ede-e7d7-462c-8fa4-cab8919da8fe admin admin] PATH_INFO: `/v3/projects/9bc60b330b2b49f7a4fab36f03eeb4ed/users/4680e1478c024aa0b0f08a7a81074eb5/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:08.198609 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b699ede-e7d7-462c-8fa4-cab8919da8fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bc60b330b2b49f7a4fab36f03eeb4ed, user_id=4680e1478c024aa0b0f08a7a81074eb5, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:08.201310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b699ede-e7d7-462c-8fa4-cab8919da8fe admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:08.300641 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1441/2875] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 10:18:08 2022] PUT /identity/v3/projects/9bc60b330b2b49f7a4fab36f03eeb4ed/users/4680e1478c024aa0b0f08a7a81074eb5/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:18:08.311061 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a961d-ed47-4815-89e6-353ca2e1fc2a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:08.311373 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a961d-ed47-4815-89e6-353ca2e1fc2a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:08.311653 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a961d-ed47-4815-89e6-353ca2e1fc2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:08.339152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-850a961d-ed47-4815-89e6-353ca2e1fc2a None None] MFA Rules not processed for user `4680e1478c024aa0b0f08a7a81074eb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:18:08.340305 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-850a961d-ed47-4815-89e6-353ca2e1fc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:08.404683 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:08.409566 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1435/2876] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:18:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:18:08.420867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b699ede-e7d7-462c-8fa4-cab8919da8fe admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:08.422758 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6b699ede-e7d7-462c-8fa4-cab8919da8fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:08.503374 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b699ede-e7d7-462c-8fa4-cab8919da8fe admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:08.509609 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0af0ef5-edda-400a-bf1b-e742bf22853f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:08.510215 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af0ef5-edda-400a-bf1b-e742bf22853f service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:08.510492 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af0ef5-edda-400a-bf1b-e742bf22853f service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:08.510906 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af0ef5-edda-400a-bf1b-e742bf22853f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:08.518645 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0af0ef5-edda-400a-bf1b-e742bf22853f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:08.520221 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0af0ef5-edda-400a-bf1b-e742bf22853f service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:08.529644 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1442/2877] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:18:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:18:08.733495 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a961d-ed47-4815-89e6-353ca2e1fc2a None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:08.804855 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a961d-ed47-4815-89e6-353ca2e1fc2a None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:08.809352 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1857e-8c9a-4867-a3e3-2ab907afd47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:08.810006 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1857e-8c9a-4867-a3e3-2ab907afd47e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:08.810287 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1857e-8c9a-4867-a3e3-2ab907afd47e admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:08.810593 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1857e-8c9a-4867-a3e3-2ab907afd47e admin admin] PATH_INFO: `/v3/users/4680e1478c024aa0b0f08a7a81074eb5` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:08.811765 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e1857e-8c9a-4867-a3e3-2ab907afd47e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4680e1478c024aa0b0f08a7a81074eb5)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:08.813857 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e1857e-8c9a-4867-a3e3-2ab907afd47e admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:08.965425 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-18e1857e-8c9a-4867-a3e3-2ab907afd47e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4680e1478c024aa0b0f08a7a81074eb5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:09.020141 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-18e1857e-8c9a-4867-a3e3-2ab907afd47e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4680e1478c024aa0b0f08a7a81074eb5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:09.020812 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-18e1857e-8c9a-4867-a3e3-2ab907afd47e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4680e1478c024aa0b0f08a7a81074eb5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:09.021438 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-18e1857e-8c9a-4867-a3e3-2ab907afd47e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4680e1478c024aa0b0f08a7a81074eb5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:09.033754 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-18e1857e-8c9a-4867-a3e3-2ab907afd47e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4680e1478c024aa0b0f08a7a81074eb5'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:09.046120 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1436/2878] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 10:18:08 2022] DELETE /identity/v3/users/4680e1478c024aa0b0f08a7a81074eb5 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:18:09.050643 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342a53c-94a1-4ee1-9902-f81c0d0aff21 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:09.051117 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342a53c-94a1-4ee1-9902-f81c0d0aff21 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:09.051443 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342a53c-94a1-4ee1-9902-f81c0d0aff21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:09.071136 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5342a53c-94a1-4ee1-9902-f81c0d0aff21 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:18:09.072269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5342a53c-94a1-4ee1-9902-f81c0d0aff21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:09.132376 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1443/2879] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:18:09.142455 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1857e-8c9a-4867-a3e3-2ab907afd47e admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:09.143916 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-18e1857e-8c9a-4867-a3e3-2ab907afd47e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:09.201891 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1857e-8c9a-4867-a3e3-2ab907afd47e admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:09.206105 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd85624-ebc0-4e1a-87a4-672837647485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:09.206789 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd85624-ebc0-4e1a-87a4-672837647485 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:09.207069 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd85624-ebc0-4e1a-87a4-672837647485 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:09.207337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd85624-ebc0-4e1a-87a4-672837647485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:09.216555 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd85624-ebc0-4e1a-87a4-672837647485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:09.217978 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd85624-ebc0-4e1a-87a4-672837647485 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:09.228325 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1437/2880] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:18:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:18:09.605858 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342a53c-94a1-4ee1-9902-f81c0d0aff21 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:09.607316 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5342a53c-94a1-4ee1-9902-f81c0d0aff21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:09.621791 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342a53c-94a1-4ee1-9902-f81c0d0aff21 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:09.629031 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c6f48a-b752-4b73-bdfc-f86a255243bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:09.629897 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6f48a-b752-4b73-bdfc-f86a255243bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:09.630329 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6f48a-b752-4b73-bdfc-f86a255243bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:09.630793 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6f48a-b752-4b73-bdfc-f86a255243bd admin admin] PATH_INFO: `/v3/projects/9bc60b330b2b49f7a4fab36f03eeb4ed` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:09.633308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c6f48a-b752-4b73-bdfc-f86a255243bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bc60b330b2b49f7a4fab36f03eeb4ed)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:09.635273 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c6f48a-b752-4b73-bdfc-f86a255243bd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:09.742209 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-59c6f48a-b752-4b73-bdfc-f86a255243bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bc60b330b2b49f7a4fab36f03eeb4ed', 'request_id': 'req-59c6f48a-b752-4b73-bdfc-f86a255243bd'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:09.832190 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-59c6f48a-b752-4b73-bdfc-f86a255243bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bc60b330b2b49f7a4fab36f03eeb4ed', 'request_id': 'req-59c6f48a-b752-4b73-bdfc-f86a255243bd'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:09.834027 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:09.842503 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-59c6f48a-b752-4b73-bdfc-f86a255243bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bc60b330b2b49f7a4fab36f03eeb4ed', 'request_id': 'req-59c6f48a-b752-4b73-bdfc-f86a255243bd'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:09.856095 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-59c6f48a-b752-4b73-bdfc-f86a255243bd admin admin] The token cache is being invalidate because project 9bc60b330b2b49f7a4fab36f03eeb4ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 10:18:09.856404 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-59c6f48a-b752-4b73-bdfc-f86a255243bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:09.857044 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-59c6f48a-b752-4b73-bdfc-f86a255243bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:09.858822 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1444/2881] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 10:18:09 2022] DELETE /identity/v3/projects/9bc60b330b2b49f7a4fab36f03eeb4ed => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:18:09.863493 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57156364-a566-4859-9ac9-1e88788a2a0e None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:09.863963 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57156364-a566-4859-9ac9-1e88788a2a0e None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:09.864150 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57156364-a566-4859-9ac9-1e88788a2a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:09.888378 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-57156364-a566-4859-9ac9-1e88788a2a0e None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:18:09.889518 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-57156364-a566-4859-9ac9-1e88788a2a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:09.902972 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:09.933101 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c6f48a-b752-4b73-bdfc-f86a255243bd admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:09.934591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-59c6f48a-b752-4b73-bdfc-f86a255243bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:09.952715 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1438/2882] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:18:09.962606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dd145-b254-46a6-a6f6-9b412392faac None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:09.962920 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dd145-b254-46a6-a6f6-9b412392faac None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:09.963199 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9dd145-b254-46a6-a6f6-9b412392faac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:09.982651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-2c9dd145-b254-46a6-a6f6-9b412392faac None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:18:09.983917 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-2c9dd145-b254-46a6-a6f6-9b412392faac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:09.992446 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1439/2883] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 10:18:09.996735 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9dd145-b254-46a6-a6f6-9b412392faac None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:10.001437 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c6f48a-b752-4b73-bdfc-f86a255243bd admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:10.005695 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9dd145-b254-46a6-a6f6-9b412392faac None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:10.006102 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f126a7-0f0c-430f-810d-2cc571b74ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:10.011178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f126a7-0f0c-430f-810d-2cc571b74ee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:10.011178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f126a7-0f0c-430f-810d-2cc571b74ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:10.011178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f126a7-0f0c-430f-810d-2cc571b74ee9 admin admin] PATH_INFO: `/v3/users/f78a8d2ab1fc4e8abd13a7ec3bba33ea` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:10.011178 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f126a7-0f0c-430f-810d-2cc571b74ee9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f78a8d2ab1fc4e8abd13a7ec3bba33ea)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:10.011527 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574c96e-840a-46d7-a58d-a80ec79be91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:10.011640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f126a7-0f0c-430f-810d-2cc571b74ee9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:10.011735 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574c96e-840a-46d7-a58d-a80ec79be91f admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:10.011819 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574c96e-840a-46d7-a58d-a80ec79be91f admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:10.012065 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8574c96e-840a-46d7-a58d-a80ec79be91f admin admin] PATH_INFO: `/v3/domains` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:10.012723 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574c96e-840a-46d7-a58d-a80ec79be91f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81611) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 10:18:10.013030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574c96e-840a-46d7-a58d-a80ec79be91f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:10.014623 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8574c96e-840a-46d7-a58d-a80ec79be91f admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:10.027523 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1440/2884] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 10:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 02 10:18:10.031904 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574c96e-840a-46d7-a58d-a80ec79be91f admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:10.041609 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8574c96e-840a-46d7-a58d-a80ec79be91f admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:10.046264 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd9b52-2c73-457e-b6e9-9db6619ecbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:10.046895 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd9b52-2c73-457e-b6e9-9db6619ecbfb admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:10.047177 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd9b52-2c73-457e-b6e9-9db6619ecbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:10.047452 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd9b52-2c73-457e-b6e9-9db6619ecbfb admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:10.048101 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd9b52-2c73-457e-b6e9-9db6619ecbfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:10.049971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd9b52-2c73-457e-b6e9-9db6619ecbfb admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:10.154784 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1441/2885] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 10:18:10 2022] POST /identity/v3/projects => generated 387 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 10:18:10.158988 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd9b52-2c73-457e-b6e9-9db6619ecbfb admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:10.221427 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd9b52-2c73-457e-b6e9-9db6619ecbfb admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:10.226616 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6017e6-58a0-48b2-a5c7-ca66804040f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:10.227384 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6017e6-58a0-48b2-a5c7-ca66804040f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:10.227673 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6017e6-58a0-48b2-a5c7-ca66804040f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:10.227944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6017e6-58a0-48b2-a5c7-ca66804040f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:10.228585 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6017e6-58a0-48b2-a5c7-ca66804040f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:10.228703 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-41f126a7-0f0c-430f-810d-2cc571b74ee9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f78a8d2ab1fc4e8abd13a7ec3bba33ea'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:10.230621 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6017e6-58a0-48b2-a5c7-ca66804040f0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:10.236283 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-41f126a7-0f0c-430f-810d-2cc571b74ee9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f78a8d2ab1fc4e8abd13a7ec3bba33ea'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:10.311736 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-41f126a7-0f0c-430f-810d-2cc571b74ee9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f78a8d2ab1fc4e8abd13a7ec3bba33ea'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:10.312381 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-41f126a7-0f0c-430f-810d-2cc571b74ee9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f78a8d2ab1fc4e8abd13a7ec3bba33ea'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:10.325800 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-41f126a7-0f0c-430f-810d-2cc571b74ee9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f78a8d2ab1fc4e8abd13a7ec3bba33ea'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:10.332106 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1445/2886] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 10:18:09 2022] DELETE /identity/v3/users/f78a8d2ab1fc4e8abd13a7ec3bba33ea => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 10:18:10.336376 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2274d-4eb7-4f54-8207-f5b22e03a0ca None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:10.336664 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2274d-4eb7-4f54-8207-f5b22e03a0ca None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:10.336942 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2274d-4eb7-4f54-8207-f5b22e03a0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:10.350111 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1442/2887] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 10:18:10 2022] POST /identity/v3/users => generated 294 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 10:18:10.351456 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6017e6-58a0-48b2-a5c7-ca66804040f0 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:10.353131 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-3f6017e6-58a0-48b2-a5c7-ca66804040f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:10.356132 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-dbc2274d-4eb7-4f54-8207-f5b22e03a0ca None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:18:10.357313 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-dbc2274d-4eb7-4f54-8207-f5b22e03a0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:10.365401 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6017e6-58a0-48b2-a5c7-ca66804040f0 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:10.423970 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1446/2888] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:18:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 10:18:10.427990 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab167ced-49f4-4aa9-b6c3-b8dcfecd8f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:10.428583 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab167ced-49f4-4aa9-b6c3-b8dcfecd8f40 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:10.428863 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab167ced-49f4-4aa9-b6c3-b8dcfecd8f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:10.429133 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab167ced-49f4-4aa9-b6c3-b8dcfecd8f40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:10.430465 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab167ced-49f4-4aa9-b6c3-b8dcfecd8f40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:10.432086 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab167ced-49f4-4aa9-b6c3-b8dcfecd8f40 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:10.434443 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2274d-4eb7-4f54-8207-f5b22e03a0ca None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:10.436113 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-dbc2274d-4eb7-4f54-8207-f5b22e03a0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:10.442347 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:10.442889 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1443/2889] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 10:18:10.447387 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab167ced-49f4-4aa9-b6c3-b8dcfecd8f40 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:10.453529 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:10.457577 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab167ced-49f4-4aa9-b6c3-b8dcfecd8f40 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:10.462360 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ab0c3-5bc8-440e-8a25-575c2ca59cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:10.470508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ab0c3-5bc8-440e-8a25-575c2ca59cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:10.470508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ab0c3-5bc8-440e-8a25-575c2ca59cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:10.470508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ab0c3-5bc8-440e-8a25-575c2ca59cf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:10.470508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ab0c3-5bc8-440e-8a25-575c2ca59cf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:10.470508 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ab0c3-5bc8-440e-8a25-575c2ca59cf8 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:10.476463 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1444/2890] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:18:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:18:10.482565 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ab0c3-5bc8-440e-8a25-575c2ca59cf8 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:10.495848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ab0c3-5bc8-440e-8a25-575c2ca59cf8 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:10.499017 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc2274d-4eb7-4f54-8207-f5b22e03a0ca None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:10.500678 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750d759-8d3e-4fba-87fb-4ce88e4664dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:10.501442 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750d759-8d3e-4fba-87fb-4ce88e4664dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:10.501817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750d759-8d3e-4fba-87fb-4ce88e4664dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:10.502094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1750d759-8d3e-4fba-87fb-4ce88e4664dd admin admin] PATH_INFO: `/v3/projects/337226d4980a4346850c0399f41b1c77/users/12afeccf11fd4585b524fdf867bae337/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:10.502664 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b1a5d4-8219-40f6-90cb-e406e8db34a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:10.503340 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1a5d4-8219-40f6-90cb-e406e8db34a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:10.503640 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1a5d4-8219-40f6-90cb-e406e8db34a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:10.503909 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1a5d4-8219-40f6-90cb-e406e8db34a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:10.514335 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b1a5d4-8219-40f6-90cb-e406e8db34a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:10.515809 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b1a5d4-8219-40f6-90cb-e406e8db34a2 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:10.518259 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1750d759-8d3e-4fba-87fb-4ce88e4664dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=337226d4980a4346850c0399f41b1c77, user_id=12afeccf11fd4585b524fdf867bae337, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:10.520901 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1750d759-8d3e-4fba-87fb-4ce88e4664dd admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:10.525630 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1447/2891] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:18:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 10:18:10.609667 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1445/2892] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 10:18:10 2022] PUT /identity/v3/projects/337226d4980a4346850c0399f41b1c77/users/12afeccf11fd4585b524fdf867bae337/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 10:18:10.620552 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5bbb5d-4f70-4b5d-bfcb-94e597c8073d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:10.620777 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5bbb5d-4f70-4b5d-bfcb-94e597c8073d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:10.620949 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5bbb5d-4f70-4b5d-bfcb-94e597c8073d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:10.649918 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-bf5bbb5d-4f70-4b5d-bfcb-94e597c8073d None None] MFA Rules not processed for user `12afeccf11fd4585b524fdf867bae337`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:18:10.651187 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bf5bbb5d-4f70-4b5d-bfcb-94e597c8073d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:10.726974 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1448/2893] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:18:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:18:10.731587 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750d759-8d3e-4fba-87fb-4ce88e4664dd admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:10.801813 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1750d759-8d3e-4fba-87fb-4ce88e4664dd admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:10.807935 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f443237-a3fd-4341-ac9e-ccad413901b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:10.808606 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f443237-a3fd-4341-ac9e-ccad413901b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:10.808886 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f443237-a3fd-4341-ac9e-ccad413901b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:10.809158 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f443237-a3fd-4341-ac9e-ccad413901b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:10.809976 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f443237-a3fd-4341-ac9e-ccad413901b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:10.811975 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f443237-a3fd-4341-ac9e-ccad413901b1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:10.902647 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1446/2894] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 10:18:10 2022] POST /identity/v3/projects => generated 387 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:18:10.906957 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5bbb5d-4f70-4b5d-bfcb-94e597c8073d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:10.919125 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5bbb5d-4f70-4b5d-bfcb-94e597c8073d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:10.926145 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f443237-a3fd-4341-ac9e-ccad413901b1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:10.934204 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-1f443237-a3fd-4341-ac9e-ccad413901b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:11.002978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d4d1a-3ecb-40cc-88ea-8801d0e992dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:11.003600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d4d1a-3ecb-40cc-88ea-8801d0e992dd admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:11.003883 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d4d1a-3ecb-40cc-88ea-8801d0e992dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:11.004184 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d4d1a-3ecb-40cc-88ea-8801d0e992dd admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:11.004848 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d4d1a-3ecb-40cc-88ea-8801d0e992dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:11.006779 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d4d1a-3ecb-40cc-88ea-8801d0e992dd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:11.011426 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f443237-a3fd-4341-ac9e-ccad413901b1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:11.016963 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c595058-bfb9-4afd-8d31-e3fb2fc14b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:11.018050 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c595058-bfb9-4afd-8d31-e3fb2fc14b70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:11.018670 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c595058-bfb9-4afd-8d31-e3fb2fc14b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:11.019031 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c595058-bfb9-4afd-8d31-e3fb2fc14b70 admin admin] PATH_INFO: `/v3/projects/068958eb674446f1923d4e79d782cfab` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:11.020394 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c595058-bfb9-4afd-8d31-e3fb2fc14b70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=068958eb674446f1923d4e79d782cfab)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:11.022299 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c595058-bfb9-4afd-8d31-e3fb2fc14b70 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:11.129099 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1449/2895] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 10:18:10 2022] POST /identity/v3/users => generated 300 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:18:11.132998 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d4d1a-3ecb-40cc-88ea-8801d0e992dd admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:11.143203 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d4d1a-3ecb-40cc-88ea-8801d0e992dd admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:11.149606 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b54c6-0321-4e87-86ea-0bf93dea7ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:11.150857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b54c6-0321-4e87-86ea-0bf93dea7ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:11.150857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b54c6-0321-4e87-86ea-0bf93dea7ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:11.150857 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b54c6-0321-4e87-86ea-0bf93dea7ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:11.151591 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874b54c6-0321-4e87-86ea-0bf93dea7ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:11.153184 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874b54c6-0321-4e87-86ea-0bf93dea7ad8 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:11.163731 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1450/2896] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 10:18:11.167178 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8c595058-bfb9-4afd-8d31-e3fb2fc14b70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '068958eb674446f1923d4e79d782cfab', 'request_id': 'req-8c595058-bfb9-4afd-8d31-e3fb2fc14b70'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:11.173944 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b54c6-0321-4e87-86ea-0bf93dea7ad8 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:11.174101 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:11.174101 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8c595058-bfb9-4afd-8d31-e3fb2fc14b70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '068958eb674446f1923d4e79d782cfab', 'request_id': 'req-8c595058-bfb9-4afd-8d31-e3fb2fc14b70'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:11.178819 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b54c6-0321-4e87-86ea-0bf93dea7ad8 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:11.180574 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8c595058-bfb9-4afd-8d31-e3fb2fc14b70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '068958eb674446f1923d4e79d782cfab', 'request_id': 'req-8c595058-bfb9-4afd-8d31-e3fb2fc14b70'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:11.191117 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:11.238907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a36397-31bd-4a6e-a8ee-fdd40ff5298c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:11.239558 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a36397-31bd-4a6e-a8ee-fdd40ff5298c admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:11.239842 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a36397-31bd-4a6e-a8ee-fdd40ff5298c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:11.240112 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a36397-31bd-4a6e-a8ee-fdd40ff5298c admin admin] PATH_INFO: `/v3/projects/bef70c36e4b049b0949cfe73dd3c43b1/users/a5edc9756fcb456ea882a30fdc8cd538/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:11.254284 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a36397-31bd-4a6e-a8ee-fdd40ff5298c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bef70c36e4b049b0949cfe73dd3c43b1, user_id=a5edc9756fcb456ea882a30fdc8cd538, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:11.256938 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a36397-31bd-4a6e-a8ee-fdd40ff5298c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:11.284925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8c595058-bfb9-4afd-8d31-e3fb2fc14b70 admin admin] The token cache is being invalidate because project 068958eb674446f1923d4e79d782cfab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 10:18:11.285216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8c595058-bfb9-4afd-8d31-e3fb2fc14b70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:11.286502 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-8c595058-bfb9-4afd-8d31-e3fb2fc14b70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:11.289729 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1447/2897] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 10:18:10 2022] DELETE /identity/v3/projects/068958eb674446f1923d4e79d782cfab => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 10:18:11.293925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c595058-bfb9-4afd-8d31-e3fb2fc14b70 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:11.295273 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-8c595058-bfb9-4afd-8d31-e3fb2fc14b70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:11.312082 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c595058-bfb9-4afd-8d31-e3fb2fc14b70 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:11.318258 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cef383-4380-4d13-bd3b-4b28f9e0a823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:11.318923 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cef383-4380-4d13-bd3b-4b28f9e0a823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:11.319201 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cef383-4380-4d13-bd3b-4b28f9e0a823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:11.319517 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cef383-4380-4d13-bd3b-4b28f9e0a823 admin admin] PATH_INFO: `/v3/users/f66a019577c54cababecaa155e04e82b` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:11.323013 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cef383-4380-4d13-bd3b-4b28f9e0a823 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f66a019577c54cababecaa155e04e82b)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:11.324912 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cef383-4380-4d13-bd3b-4b28f9e0a823 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:11.341786 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1451/2898] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 10:18:11 2022] PUT /identity/v3/projects/bef70c36e4b049b0949cfe73dd3c43b1/users/a5edc9756fcb456ea882a30fdc8cd538/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 10:18:11.345677 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a36397-31bd-4a6e-a8ee-fdd40ff5298c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:11.347382 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d9a36397-31bd-4a6e-a8ee-fdd40ff5298c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:11.410401 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a36397-31bd-4a6e-a8ee-fdd40ff5298c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:11.416013 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4a3d5-89ca-420d-8648-562440730aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:11.416624 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4a3d5-89ca-420d-8648-562440730aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:11.416908 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4a3d5-89ca-420d-8648-562440730aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:11.417187 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4a3d5-89ca-420d-8648-562440730aa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:11.417832 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4a3d5-89ca-420d-8648-562440730aa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:11.419517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4a3d5-89ca-420d-8648-562440730aa7 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:11.429767 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1452/2899] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 10:18:11.433911 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4a3d5-89ca-420d-8648-562440730aa7 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:11.444799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4a3d5-89ca-420d-8648-562440730aa7 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:11.449726 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f12727-b4d0-402e-b41c-0a37d96122b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:11.450366 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f12727-b4d0-402e-b41c-0a37d96122b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:11.450648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f12727-b4d0-402e-b41c-0a37d96122b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:11.450964 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f12727-b4d0-402e-b41c-0a37d96122b0 admin admin] PATH_INFO: `/v3/projects/bef70c36e4b049b0949cfe73dd3c43b1/users/a5edc9756fcb456ea882a30fdc8cd538/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:11.452876 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f12727-b4d0-402e-b41c-0a37d96122b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bef70c36e4b049b0949cfe73dd3c43b1, user_id=a5edc9756fcb456ea882a30fdc8cd538, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:11.455569 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f12727-b4d0-402e-b41c-0a37d96122b0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:11.466191 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1453/2900] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 10:18:11 2022] PUT /identity/v3/projects/bef70c36e4b049b0949cfe73dd3c43b1/users/a5edc9756fcb456ea882a30fdc8cd538/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 10:18:11.476159 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c084b2d-c89c-4061-8e31-1e3d43bac0c9 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:11.484949 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c084b2d-c89c-4061-8e31-1e3d43bac0c9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:11.484949 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c084b2d-c89c-4061-8e31-1e3d43bac0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:11.504398 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5c084b2d-c89c-4061-8e31-1e3d43bac0c9 None None] MFA Rules not processed for user `a5edc9756fcb456ea882a30fdc8cd538`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:18:11.505605 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5c084b2d-c89c-4061-8e31-1e3d43bac0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:11.566847 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d5cef383-4380-4d13-bd3b-4b28f9e0a823 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f66a019577c54cababecaa155e04e82b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:11.580275 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1454/2901] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:18:11 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 10:18:11.592319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c084b2d-c89c-4061-8e31-1e3d43bac0c9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:11.593780 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5c084b2d-c89c-4061-8e31-1e3d43bac0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:11.609427 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:11.631893 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d5cef383-4380-4d13-bd3b-4b28f9e0a823 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f66a019577c54cababecaa155e04e82b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:11.633146 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d5cef383-4380-4d13-bd3b-4b28f9e0a823 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f66a019577c54cababecaa155e04e82b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:11.633908 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d5cef383-4380-4d13-bd3b-4b28f9e0a823 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f66a019577c54cababecaa155e04e82b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:11.648201 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d5cef383-4380-4d13-bd3b-4b28f9e0a823 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f66a019577c54cababecaa155e04e82b'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:11.653438 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c084b2d-c89c-4061-8e31-1e3d43bac0c9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:11.657088 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdbeef6-6640-4095-9866-08c41aff3969 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:11.657702 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdbeef6-6640-4095-9866-08c41aff3969 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:11.657984 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdbeef6-6640-4095-9866-08c41aff3969 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:11.658261 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdbeef6-6640-4095-9866-08c41aff3969 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:11.660426 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2cdbeef6-6640-4095-9866-08c41aff3969 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:11.661168 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1448/2902] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 10:18:11 2022] DELETE /identity/v3/users/f66a019577c54cababecaa155e04e82b => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:18:11.673357 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdbeef6-6640-4095-9866-08c41aff3969 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:11.674815 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdbeef6-6640-4095-9866-08c41aff3969 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:11.686519 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1455/2903] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:18:11 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 10:18:11.718121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cef383-4380-4d13-bd3b-4b28f9e0a823 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:11.719608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d5cef383-4380-4d13-bd3b-4b28f9e0a823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:11.792740 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cef383-4380-4d13-bd3b-4b28f9e0a823 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:11.797470 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5c9609-0b6f-4c1f-a40d-3fc829dc1c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:11.798099 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c9609-0b6f-4c1f-a40d-3fc829dc1c11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:11.798376 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c9609-0b6f-4c1f-a40d-3fc829dc1c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:11.798649 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c9609-0b6f-4c1f-a40d-3fc829dc1c11 admin admin] PATH_INFO: `/v3/users/12afeccf11fd4585b524fdf867bae337` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:11.799795 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c9609-0b6f-4c1f-a40d-3fc829dc1c11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12afeccf11fd4585b524fdf867bae337)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:11.801697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5c9609-0b6f-4c1f-a40d-3fc829dc1c11 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:12.005973 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ad5c9609-0b6f-4c1f-a40d-3fc829dc1c11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12afeccf11fd4585b524fdf867bae337'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:12.079152 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ad5c9609-0b6f-4c1f-a40d-3fc829dc1c11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12afeccf11fd4585b524fdf867bae337'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:12.079889 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ad5c9609-0b6f-4c1f-a40d-3fc829dc1c11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12afeccf11fd4585b524fdf867bae337'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:12.080006 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdbeef6-6640-4095-9866-08c41aff3969 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:12.081708 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2cdbeef6-6640-4095-9866-08c41aff3969 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:12.083245 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ad5c9609-0b6f-4c1f-a40d-3fc829dc1c11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12afeccf11fd4585b524fdf867bae337'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:12.090030 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=78,keystone:INSERT=3,keystone:DELETE=19 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:12.102292 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-ad5c9609-0b6f-4c1f-a40d-3fc829dc1c11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12afeccf11fd4585b524fdf867bae337'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:12.116583 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1449/2904] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 10:18:11 2022] DELETE /identity/v3/users/12afeccf11fd4585b524fdf867bae337 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:18:12.122371 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995dce72-3bc7-4b83-84d6-80096f8ef146 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:12.122868 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995dce72-3bc7-4b83-84d6-80096f8ef146 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:12.123160 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995dce72-3bc7-4b83-84d6-80096f8ef146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:12.145031 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-995dce72-3bc7-4b83-84d6-80096f8ef146 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:18:12.146528 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-995dce72-3bc7-4b83-84d6-80096f8ef146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:12.165058 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdbeef6-6640-4095-9866-08c41aff3969 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:12.213913 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1450/2905] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:18:12.223844 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e498bf-93fe-4822-a479-d12616395286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:12.224021 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995dce72-3bc7-4b83-84d6-80096f8ef146 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:12.224568 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e498bf-93fe-4822-a479-d12616395286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:12.224854 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e498bf-93fe-4822-a479-d12616395286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:12.225131 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e498bf-93fe-4822-a479-d12616395286 admin admin] PATH_INFO: `/v3/projects/f41838411bab42e7981f4f1f73104cc2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:12.225234 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-995dce72-3bc7-4b83-84d6-80096f8ef146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:12.226849 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e498bf-93fe-4822-a479-d12616395286 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f41838411bab42e7981f4f1f73104cc2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:12.228876 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e498bf-93fe-4822-a479-d12616395286 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:12.282540 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995dce72-3bc7-4b83-84d6-80096f8ef146 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:12.286192 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca6946-c6ae-43dc-90fb-69fde114574a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:12.286833 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca6946-c6ae-43dc-90fb-69fde114574a service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:12.287140 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca6946-c6ae-43dc-90fb-69fde114574a service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:12.287447 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca6946-c6ae-43dc-90fb-69fde114574a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:12.296938 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca6946-c6ae-43dc-90fb-69fde114574a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:12.298360 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca6946-c6ae-43dc-90fb-69fde114574a service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:12.308415 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1451/2906] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:18:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 02 10:18:12.335213 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-45e498bf-93fe-4822-a479-d12616395286 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f41838411bab42e7981f4f1f73104cc2', 'request_id': 'req-45e498bf-93fe-4822-a479-d12616395286'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:12.420829 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-45e498bf-93fe-4822-a479-d12616395286 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f41838411bab42e7981f4f1f73104cc2', 'request_id': 'req-45e498bf-93fe-4822-a479-d12616395286'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:12.423682 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:12.427847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-45e498bf-93fe-4822-a479-d12616395286 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f41838411bab42e7981f4f1f73104cc2', 'request_id': 'req-45e498bf-93fe-4822-a479-d12616395286'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:12.439279 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-45e498bf-93fe-4822-a479-d12616395286 admin admin] The token cache is being invalidate because project f41838411bab42e7981f4f1f73104cc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 10:18:12.439564 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-45e498bf-93fe-4822-a479-d12616395286 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:12.440197 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-45e498bf-93fe-4822-a479-d12616395286 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:12.441663 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1456/2907] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 10:18:12 2022] DELETE /identity/v3/projects/f41838411bab42e7981f4f1f73104cc2 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 10:18:12.445651 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca6946-c6ae-43dc-90fb-69fde114574a service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:12.454684 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f0ca6946-c6ae-43dc-90fb-69fde114574a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:12.459812 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca6946-c6ae-43dc-90fb-69fde114574a service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:12.518993 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942c4cf-0cae-4cb5-af92-2cee5c99b5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:12.519641 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942c4cf-0cae-4cb5-af92-2cee5c99b5ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:12.519923 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942c4cf-0cae-4cb5-af92-2cee5c99b5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:12.520344 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942c4cf-0cae-4cb5-af92-2cee5c99b5ec admin admin] PATH_INFO: `/v3/users/f3d74697fa854f7a962af2b19c7b7ba0` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:12.521417 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e942c4cf-0cae-4cb5-af92-2cee5c99b5ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3d74697fa854f7a962af2b19c7b7ba0)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:12.523375 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e942c4cf-0cae-4cb5-af92-2cee5c99b5ec admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:12.593314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=5,keystone:SELECT=95 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:12.738043 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e942c4cf-0cae-4cb5-af92-2cee5c99b5ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3d74697fa854f7a962af2b19c7b7ba0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:12.753922 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e498bf-93fe-4822-a479-d12616395286 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:12.755479 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-45e498bf-93fe-4822-a479-d12616395286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:12.768228 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e498bf-93fe-4822-a479-d12616395286 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:12.773869 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f028d7-7507-48f1-ac05-5434b874d9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:12.774397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f028d7-7507-48f1-ac05-5434b874d9e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:12.774580 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f028d7-7507-48f1-ac05-5434b874d9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:12.774887 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f028d7-7507-48f1-ac05-5434b874d9e1 admin admin] PATH_INFO: `/v3/projects/337226d4980a4346850c0399f41b1c77` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:12.777613 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f028d7-7507-48f1-ac05-5434b874d9e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=337226d4980a4346850c0399f41b1c77)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:12.782912 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f028d7-7507-48f1-ac05-5434b874d9e1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:12.809031 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e942c4cf-0cae-4cb5-af92-2cee5c99b5ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3d74697fa854f7a962af2b19c7b7ba0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:12.809681 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e942c4cf-0cae-4cb5-af92-2cee5c99b5ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3d74697fa854f7a962af2b19c7b7ba0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:12.810333 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e942c4cf-0cae-4cb5-af92-2cee5c99b5ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3d74697fa854f7a962af2b19c7b7ba0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:12.823418 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-e942c4cf-0cae-4cb5-af92-2cee5c99b5ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3d74697fa854f7a962af2b19c7b7ba0'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:12.836717 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1452/2908] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 10:18:12 2022] DELETE /identity/v3/users/f3d74697fa854f7a962af2b19c7b7ba0 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:18:12.892532 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-92f028d7-7507-48f1-ac05-5434b874d9e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '337226d4980a4346850c0399f41b1c77', 'request_id': 'req-92f028d7-7507-48f1-ac05-5434b874d9e1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:12.968322 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-92f028d7-7507-48f1-ac05-5434b874d9e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '337226d4980a4346850c0399f41b1c77', 'request_id': 'req-92f028d7-7507-48f1-ac05-5434b874d9e1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:12.974274 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-92f028d7-7507-48f1-ac05-5434b874d9e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '337226d4980a4346850c0399f41b1c77', 'request_id': 'req-92f028d7-7507-48f1-ac05-5434b874d9e1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:12.985573 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-92f028d7-7507-48f1-ac05-5434b874d9e1 admin admin] The token cache is being invalidate because project 337226d4980a4346850c0399f41b1c77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 10:18:12.985856 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-92f028d7-7507-48f1-ac05-5434b874d9e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:12.986478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-92f028d7-7507-48f1-ac05-5434b874d9e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:12.987946 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1457/2909] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 10:18:12 2022] DELETE /identity/v3/projects/337226d4980a4346850c0399f41b1c77 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 10:18:12.991944 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942c4cf-0cae-4cb5-af92-2cee5c99b5ec admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:12.993340 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-e942c4cf-0cae-4cb5-af92-2cee5c99b5ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:13.059354 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942c4cf-0cae-4cb5-af92-2cee5c99b5ec admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:13.120162 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b71fc-f473-46f5-b67c-14212d43e9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:13.120797 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b71fc-f473-46f5-b67c-14212d43e9b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:13.121073 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b71fc-f473-46f5-b67c-14212d43e9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:13.121363 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b71fc-f473-46f5-b67c-14212d43e9b3 admin admin] PATH_INFO: `/v3/users/a5edc9756fcb456ea882a30fdc8cd538` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:13.122509 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b71fc-f473-46f5-b67c-14212d43e9b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5edc9756fcb456ea882a30fdc8cd538)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:13.124443 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b71fc-f473-46f5-b67c-14212d43e9b3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:13.292920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f028d7-7507-48f1-ac05-5434b874d9e1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:13.292920 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-92f028d7-7507-48f1-ac05-5434b874d9e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:13.308757 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f028d7-7507-48f1-ac05-5434b874d9e1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:13.313494 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d300f2-6ff1-42c8-b8aa-ad11f8a88bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:13.315208 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d300f2-6ff1-42c8-b8aa-ad11f8a88bbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:13.315501 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d300f2-6ff1-42c8-b8aa-ad11f8a88bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:13.315833 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d300f2-6ff1-42c8-b8aa-ad11f8a88bbc admin admin] PATH_INFO: `/v3/projects/f0fd972f534545eb86cac5c6aad18bc3` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:13.316895 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d300f2-6ff1-42c8-b8aa-ad11f8a88bbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0fd972f534545eb86cac5c6aad18bc3)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:13.318816 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d300f2-6ff1-42c8-b8aa-ad11f8a88bbc admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:13.337526 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-378b71fc-f473-46f5-b67c-14212d43e9b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5edc9756fcb456ea882a30fdc8cd538'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:13.339302 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:13.449174 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-378b71fc-f473-46f5-b67c-14212d43e9b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5edc9756fcb456ea882a30fdc8cd538'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:13.449799 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-378b71fc-f473-46f5-b67c-14212d43e9b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5edc9756fcb456ea882a30fdc8cd538'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:13.450404 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-378b71fc-f473-46f5-b67c-14212d43e9b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5edc9756fcb456ea882a30fdc8cd538'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:13.464062 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-378b71fc-f473-46f5-b67c-14212d43e9b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5edc9756fcb456ea882a30fdc8cd538'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:13.476627 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1453/2910] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 10:18:12 2022] DELETE /identity/v3/users/a5edc9756fcb456ea882a30fdc8cd538 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:18:13.478880 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a1d300f2-6ff1-42c8-b8aa-ad11f8a88bbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0fd972f534545eb86cac5c6aad18bc3', 'request_id': 'req-a1d300f2-6ff1-42c8-b8aa-ad11f8a88bbc'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:13.574635 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a1d300f2-6ff1-42c8-b8aa-ad11f8a88bbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0fd972f534545eb86cac5c6aad18bc3', 'request_id': 'req-a1d300f2-6ff1-42c8-b8aa-ad11f8a88bbc'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:13.580749 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a1d300f2-6ff1-42c8-b8aa-ad11f8a88bbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0fd972f534545eb86cac5c6aad18bc3', 'request_id': 'req-a1d300f2-6ff1-42c8-b8aa-ad11f8a88bbc'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:13.592071 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a1d300f2-6ff1-42c8-b8aa-ad11f8a88bbc admin admin] The token cache is being invalidate because project f0fd972f534545eb86cac5c6aad18bc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 10:18:13.592356 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a1d300f2-6ff1-42c8-b8aa-ad11f8a88bbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:13.593001 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a1d300f2-6ff1-42c8-b8aa-ad11f8a88bbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:13.594500 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1458/2911] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 10:18:13 2022] DELETE /identity/v3/projects/f0fd972f534545eb86cac5c6aad18bc3 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 10:18:13.896705 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b71fc-f473-46f5-b67c-14212d43e9b3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:13.898190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-378b71fc-f473-46f5-b67c-14212d43e9b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:13.962654 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b71fc-f473-46f5-b67c-14212d43e9b3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:13.967194 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ad0713-1624-42e2-88df-055ca9d6dbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:13.967862 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad0713-1624-42e2-88df-055ca9d6dbaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:13.968139 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad0713-1624-42e2-88df-055ca9d6dbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:13.968415 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ad0713-1624-42e2-88df-055ca9d6dbaa admin admin] PATH_INFO: `/v3/projects/bef70c36e4b049b0949cfe73dd3c43b1` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:13.969522 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad0713-1624-42e2-88df-055ca9d6dbaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=bef70c36e4b049b0949cfe73dd3c43b1)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:13.971670 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ad0713-1624-42e2-88df-055ca9d6dbaa admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:14.066109 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-62ad0713-1624-42e2-88df-055ca9d6dbaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bef70c36e4b049b0949cfe73dd3c43b1', 'request_id': 'req-62ad0713-1624-42e2-88df-055ca9d6dbaa'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:14.071775 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-62ad0713-1624-42e2-88df-055ca9d6dbaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bef70c36e4b049b0949cfe73dd3c43b1', 'request_id': 'req-62ad0713-1624-42e2-88df-055ca9d6dbaa'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:14.077836 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-62ad0713-1624-42e2-88df-055ca9d6dbaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bef70c36e4b049b0949cfe73dd3c43b1', 'request_id': 'req-62ad0713-1624-42e2-88df-055ca9d6dbaa'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:14.121774 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-62ad0713-1624-42e2-88df-055ca9d6dbaa admin admin] The token cache is being invalidate because project bef70c36e4b049b0949cfe73dd3c43b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 10:18:14.121997 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-62ad0713-1624-42e2-88df-055ca9d6dbaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:14.122628 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-62ad0713-1624-42e2-88df-055ca9d6dbaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:14.124183 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1454/2912] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 10:18:13 2022] DELETE /identity/v3/projects/bef70c36e4b049b0949cfe73dd3c43b1 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:18:14.129056 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c728f4-a389-4e68-a113-b366e9fe326f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:14.129340 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c728f4-a389-4e68-a113-b366e9fe326f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:14.129614 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c728f4-a389-4e68-a113-b366e9fe326f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:14.149786 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-07c728f4-a389-4e68-a113-b366e9fe326f None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:18:14.151002 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-07c728f4-a389-4e68-a113-b366e9fe326f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:14.196295 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:14.213130 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1459/2913] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:18:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:18:14.222025 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504151f3-a817-438f-9cc1-0e40757c0f69 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:14.222379 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504151f3-a817-438f-9cc1-0e40757c0f69 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:14.222594 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504151f3-a817-438f-9cc1-0e40757c0f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:14.243141 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-504151f3-a817-438f-9cc1-0e40757c0f69 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:18:14.244382 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-504151f3-a817-438f-9cc1-0e40757c0f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:14.252349 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1455/2914] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:18:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:18:14.256493 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c728f4-a389-4e68-a113-b366e9fe326f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:14.265935 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c728f4-a389-4e68-a113-b366e9fe326f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:14.270683 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6c8b8-8f46-4783-a14f-57b15a046073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:14.271572 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6c8b8-8f46-4783-a14f-57b15a046073 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:14.272046 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6c8b8-8f46-4783-a14f-57b15a046073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:14.272546 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f6c8b8-8f46-4783-a14f-57b15a046073 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:14.273729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6c8b8-8f46-4783-a14f-57b15a046073 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 10:18:14.274035 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6c8b8-8f46-4783-a14f-57b15a046073 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:14.275634 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f6c8b8-8f46-4783-a14f-57b15a046073 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:14.286882 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1460/2915] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 10:18:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 10:18:14.291739 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504151f3-a817-438f-9cc1-0e40757c0f69 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:14.300706 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504151f3-a817-438f-9cc1-0e40757c0f69 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:14.305270 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1502e6f-3db4-43db-be1b-ae2bde5af1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:14.305853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1502e6f-3db4-43db-be1b-ae2bde5af1b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:14.306131 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1502e6f-3db4-43db-be1b-ae2bde5af1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:14.306398 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1502e6f-3db4-43db-be1b-ae2bde5af1b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:14.307070 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1502e6f-3db4-43db-be1b-ae2bde5af1b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:14.308935 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1502e6f-3db4-43db-be1b-ae2bde5af1b0 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:14.410579 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1456/2916] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 10:18:14 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:18:14.415173 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6c8b8-8f46-4783-a14f-57b15a046073 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:14.425354 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f6c8b8-8f46-4783-a14f-57b15a046073 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:14.480687 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d02d93-0646-47bd-b1e1-330c34f0ed1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:14.481309 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d02d93-0646-47bd-b1e1-330c34f0ed1a admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:14.481556 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d02d93-0646-47bd-b1e1-330c34f0ed1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:14.481829 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d02d93-0646-47bd-b1e1-330c34f0ed1a admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:14.482448 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d02d93-0646-47bd-b1e1-330c34f0ed1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:14.484390 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d02d93-0646-47bd-b1e1-330c34f0ed1a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:14.568624 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1461/2917] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 10:18:14 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:18:14.573416 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1502e6f-3db4-43db-be1b-ae2bde5af1b0 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:14.584136 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1502e6f-3db4-43db-be1b-ae2bde5af1b0 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:14.590428 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b7683-9507-428b-9c95-be21d8102319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:14.591069 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b7683-9507-428b-9c95-be21d8102319 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:14.591355 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b7683-9507-428b-9c95-be21d8102319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:14.591631 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b7683-9507-428b-9c95-be21d8102319 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:14.592277 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b7683-9507-428b-9c95-be21d8102319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:14.593930 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b7683-9507-428b-9c95-be21d8102319 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:14.604114 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1457/2918] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:18:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:18:14.607891 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d02d93-0646-47bd-b1e1-330c34f0ed1a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:14.617731 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d02d93-0646-47bd-b1e1-330c34f0ed1a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:14.622327 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959bb9e5-bac0-4322-b6f4-4cae70500b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:14.622949 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959bb9e5-bac0-4322-b6f4-4cae70500b35 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:14.623233 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959bb9e5-bac0-4322-b6f4-4cae70500b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:14.623512 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959bb9e5-bac0-4322-b6f4-4cae70500b35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:14.624151 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959bb9e5-bac0-4322-b6f4-4cae70500b35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:14.625729 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959bb9e5-bac0-4322-b6f4-4cae70500b35 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:14.636701 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1462/2919] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:18:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:18:14.640533 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b7683-9507-428b-9c95-be21d8102319 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:14.651441 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b7683-9507-428b-9c95-be21d8102319 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:14.656281 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdea256-db09-458f-aa9b-eb9669cbdcb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:14.656876 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdea256-db09-458f-aa9b-eb9669cbdcb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:14.657216 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdea256-db09-458f-aa9b-eb9669cbdcb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:14.657513 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdea256-db09-458f-aa9b-eb9669cbdcb9 admin admin] PATH_INFO: `/v3/projects/5761021f7e384897b1f1ec090be1c0a8/users/2f4e80b54ae14d23877250017c9606b0/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:14.669297 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:INSERT=3,keystone:DELETE=10,keystone:SELECT=87 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:14.672379 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdea256-db09-458f-aa9b-eb9669cbdcb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5761021f7e384897b1f1ec090be1c0a8, user_id=2f4e80b54ae14d23877250017c9606b0, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:14.675050 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdea256-db09-458f-aa9b-eb9669cbdcb9 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:14.753710 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1458/2920] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 10:18:14 2022] PUT /identity/v3/projects/5761021f7e384897b1f1ec090be1c0a8/users/2f4e80b54ae14d23877250017c9606b0/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:18:14.764497 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34654ecf-e702-4612-bdd7-24e7d47a8af4 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:14.765189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34654ecf-e702-4612-bdd7-24e7d47a8af4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:14.765651 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34654ecf-e702-4612-bdd7-24e7d47a8af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:14.797987 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-34654ecf-e702-4612-bdd7-24e7d47a8af4 None None] MFA Rules not processed for user `2f4e80b54ae14d23877250017c9606b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:18:14.799242 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-34654ecf-e702-4612-bdd7-24e7d47a8af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:14.869220 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1463/2921] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:18:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 10:18:14.880213 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdea256-db09-458f-aa9b-eb9669cbdcb9 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:14.881985 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-abdea256-db09-458f-aa9b-eb9669cbdcb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:14.941783 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdea256-db09-458f-aa9b-eb9669cbdcb9 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:14.945586 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105733f-007e-4332-ba6c-44c46bbc65a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:14.946456 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105733f-007e-4332-ba6c-44c46bbc65a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:14.947483 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105733f-007e-4332-ba6c-44c46bbc65a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:14.947862 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105733f-007e-4332-ba6c-44c46bbc65a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:14.957409 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3105733f-007e-4332-ba6c-44c46bbc65a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:14.958853 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3105733f-007e-4332-ba6c-44c46bbc65a1 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:14.967902 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1459/2922] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:18:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:18:24.865908 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=75,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:24.963657 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=29,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:25.611459 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34654ecf-e702-4612-bdd7-24e7d47a8af4 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:25.673848 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34654ecf-e702-4612-bdd7-24e7d47a8af4 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:25.680887 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf946778-512f-42e7-ab34-5c3bc6e60813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:25.681763 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf946778-512f-42e7-ab34-5c3bc6e60813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:25.682058 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf946778-512f-42e7-ab34-5c3bc6e60813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:25.682410 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf946778-512f-42e7-ab34-5c3bc6e60813 admin admin] PATH_INFO: `/v3/users/2f4e80b54ae14d23877250017c9606b0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:25.683586 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf946778-512f-42e7-ab34-5c3bc6e60813 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f4e80b54ae14d23877250017c9606b0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:25.685499 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf946778-512f-42e7-ab34-5c3bc6e60813 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:25.828809 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-bf946778-512f-42e7-ab34-5c3bc6e60813 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f4e80b54ae14d23877250017c9606b0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:25.835789 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-bf946778-512f-42e7-ab34-5c3bc6e60813 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f4e80b54ae14d23877250017c9606b0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:25.894405 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-bf946778-512f-42e7-ab34-5c3bc6e60813 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f4e80b54ae14d23877250017c9606b0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:25.895319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-bf946778-512f-42e7-ab34-5c3bc6e60813 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f4e80b54ae14d23877250017c9606b0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:25.909470 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-bf946778-512f-42e7-ab34-5c3bc6e60813 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f4e80b54ae14d23877250017c9606b0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:25.915521 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1464/2923] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 10:18:25 2022] DELETE /identity/v3/users/2f4e80b54ae14d23877250017c9606b0 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:18:25.920828 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3c8c3-84ee-4505-a638-6b3455c6a7d4 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:25.921118 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3c8c3-84ee-4505-a638-6b3455c6a7d4 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:25.921398 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3c8c3-84ee-4505-a638-6b3455c6a7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:25.940201 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-bff3c8c3-84ee-4505-a638-6b3455c6a7d4 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:18:25.941441 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bff3c8c3-84ee-4505-a638-6b3455c6a7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:26.001558 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1460/2924] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:18:26.011877 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf946778-512f-42e7-ab34-5c3bc6e60813 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:26.013309 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bf946778-512f-42e7-ab34-5c3bc6e60813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:26.068439 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf946778-512f-42e7-ab34-5c3bc6e60813 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:26.072770 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af951476-e483-4285-9b3b-30a5773b3d0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:26.073360 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af951476-e483-4285-9b3b-30a5773b3d0f service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:26.073654 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af951476-e483-4285-9b3b-30a5773b3d0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:26.073905 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af951476-e483-4285-9b3b-30a5773b3d0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:26.082926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af951476-e483-4285-9b3b-30a5773b3d0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:26.088194 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af951476-e483-4285-9b3b-30a5773b3d0f service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:26.093940 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1465/2925] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:18:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:18:26.479239 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff3c8c3-84ee-4505-a638-6b3455c6a7d4 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:26.480671 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-bff3c8c3-84ee-4505-a638-6b3455c6a7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:26.492892 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff3c8c3-84ee-4505-a638-6b3455c6a7d4 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:26.498066 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ec4ba-c247-4209-8d6f-71ece23add30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:26.498698 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ec4ba-c247-4209-8d6f-71ece23add30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:26.499005 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ec4ba-c247-4209-8d6f-71ece23add30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:26.499322 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ec4ba-c247-4209-8d6f-71ece23add30 admin admin] PATH_INFO: `/v3/projects/5761021f7e384897b1f1ec090be1c0a8` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:26.500399 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ec4ba-c247-4209-8d6f-71ece23add30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5761021f7e384897b1f1ec090be1c0a8)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:26.502283 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ec4ba-c247-4209-8d6f-71ece23add30 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:26.597540 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-408ec4ba-c247-4209-8d6f-71ece23add30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5761021f7e384897b1f1ec090be1c0a8', 'request_id': 'req-408ec4ba-c247-4209-8d6f-71ece23add30'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:26.603090 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-408ec4ba-c247-4209-8d6f-71ece23add30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5761021f7e384897b1f1ec090be1c0a8', 'request_id': 'req-408ec4ba-c247-4209-8d6f-71ece23add30'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:26.609732 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-408ec4ba-c247-4209-8d6f-71ece23add30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5761021f7e384897b1f1ec090be1c0a8', 'request_id': 'req-408ec4ba-c247-4209-8d6f-71ece23add30'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:26.671502 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-408ec4ba-c247-4209-8d6f-71ece23add30 admin admin] The token cache is being invalidate because project 5761021f7e384897b1f1ec090be1c0a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 10:18:26.671719 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-408ec4ba-c247-4209-8d6f-71ece23add30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:26.672342 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-408ec4ba-c247-4209-8d6f-71ece23add30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:26.673861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1461/2926] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 10:18:26 2022] DELETE /identity/v3/projects/5761021f7e384897b1f1ec090be1c0a8 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 10:18:26.679002 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de00f31-c1bb-47de-88e0-8b43102e3340 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:26.679291 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de00f31-c1bb-47de-88e0-8b43102e3340 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:26.679593 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de00f31-c1bb-47de-88e0-8b43102e3340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:26.700343 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2de00f31-c1bb-47de-88e0-8b43102e3340 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:18:26.701674 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2de00f31-c1bb-47de-88e0-8b43102e3340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:26.719245 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:26.761249 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1466/2927] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:18:26.770091 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903c3df-40b0-4b09-b604-c804f5030800 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:26.771779 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903c3df-40b0-4b09-b604-c804f5030800 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:26.771779 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903c3df-40b0-4b09-b604-c804f5030800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:26.790672 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-b903c3df-40b0-4b09-b604-c804f5030800 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:18:26.792325 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b903c3df-40b0-4b09-b604-c804f5030800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:26.800163 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1462/2928] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:18:26.806867 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de00f31-c1bb-47de-88e0-8b43102e3340 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:26.814319 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de00f31-c1bb-47de-88e0-8b43102e3340 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:26.818966 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ac3d0-7c73-41f5-853c-ff02dbee4a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:26.819565 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ac3d0-7c73-41f5-853c-ff02dbee4a16 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:26.819847 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ac3d0-7c73-41f5-853c-ff02dbee4a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:26.820123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ac3d0-7c73-41f5-853c-ff02dbee4a16 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:26.820770 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ac3d0-7c73-41f5-853c-ff02dbee4a16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 10:18:26.821076 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ac3d0-7c73-41f5-853c-ff02dbee4a16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:26.822856 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ac3d0-7c73-41f5-853c-ff02dbee4a16 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:26.834811 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1467/2929] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 10:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 10:18:26.839962 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903c3df-40b0-4b09-b604-c804f5030800 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:26.849637 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903c3df-40b0-4b09-b604-c804f5030800 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:26.855096 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17913e3d-2db3-4775-9c31-e4d606c12e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:26.858519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17913e3d-2db3-4775-9c31-e4d606c12e7d admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:26.858519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17913e3d-2db3-4775-9c31-e4d606c12e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:26.858519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17913e3d-2db3-4775-9c31-e4d606c12e7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:26.858519 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17913e3d-2db3-4775-9c31-e4d606c12e7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:26.858899 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17913e3d-2db3-4775-9c31-e4d606c12e7d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:26.942215 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1463/2930] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 10:18:26 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 02 10:18:26.946235 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ac3d0-7c73-41f5-853c-ff02dbee4a16 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:26.956450 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ac3d0-7c73-41f5-853c-ff02dbee4a16 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:27.014445 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67ca00-c449-4233-8900-d63b25955d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:27.015081 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67ca00-c449-4233-8900-d63b25955d37 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:27.015361 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67ca00-c449-4233-8900-d63b25955d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:27.015631 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67ca00-c449-4233-8900-d63b25955d37 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:27.016266 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b67ca00-c449-4233-8900-d63b25955d37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:27.018159 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b67ca00-c449-4233-8900-d63b25955d37 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:27.088732 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1468/2931] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 10:18:26 2022] POST /identity/v3/users => generated 287 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:18:27.092939 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17913e3d-2db3-4775-9c31-e4d606c12e7d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:27.103454 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17913e3d-2db3-4775-9c31-e4d606c12e7d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:27.108123 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe878f-4454-4234-ac7c-4d395b9b85b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:27.110841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe878f-4454-4234-ac7c-4d395b9b85b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:27.110841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe878f-4454-4234-ac7c-4d395b9b85b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:27.110841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe878f-4454-4234-ac7c-4d395b9b85b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:27.110841 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fe878f-4454-4234-ac7c-4d395b9b85b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:27.111584 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fe878f-4454-4234-ac7c-4d395b9b85b1 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:27.122162 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1464/2932] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:18:27.126121 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67ca00-c449-4233-8900-d63b25955d37 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:27.136898 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67ca00-c449-4233-8900-d63b25955d37 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:27.143429 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63b3d3-e717-4cf9-8593-bce3093e8bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:27.143429 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63b3d3-e717-4cf9-8593-bce3093e8bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:27.143429 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63b3d3-e717-4cf9-8593-bce3093e8bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:27.143429 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63b3d3-e717-4cf9-8593-bce3093e8bd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:27.143809 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba63b3d3-e717-4cf9-8593-bce3093e8bd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:27.144830 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba63b3d3-e717-4cf9-8593-bce3093e8bd2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:27.155705 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1469/2933] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:18:27.159837 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe878f-4454-4234-ac7c-4d395b9b85b1 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:27.170290 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fe878f-4454-4234-ac7c-4d395b9b85b1 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:27.174971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940ed09-3914-41b7-885d-507d7c6d1a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:27.175613 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940ed09-3914-41b7-885d-507d7c6d1a34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:27.175890 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940ed09-3914-41b7-885d-507d7c6d1a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:27.176172 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940ed09-3914-41b7-885d-507d7c6d1a34 admin admin] PATH_INFO: `/v3/projects/b4162cb064794fc295aba1c0d39c18c4/users/7631222e250742dd847275be057e85b2/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:27.191059 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0940ed09-3914-41b7-885d-507d7c6d1a34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4162cb064794fc295aba1c0d39c18c4, user_id=7631222e250742dd847275be057e85b2, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:27.193727 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0940ed09-3914-41b7-885d-507d7c6d1a34 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:27.259802 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1465/2934] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 10:18:27 2022] PUT /identity/v3/projects/b4162cb064794fc295aba1c0d39c18c4/users/7631222e250742dd847275be057e85b2/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:18:27.273066 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc2ba5-7640-40fd-964c-ad50264d90fc None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:27.273423 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc2ba5-7640-40fd-964c-ad50264d90fc None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:27.273701 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc2ba5-7640-40fd-964c-ad50264d90fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:27.300889 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-64bc2ba5-7640-40fd-964c-ad50264d90fc None None] MFA Rules not processed for user `7631222e250742dd847275be057e85b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:18:27.302096 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-64bc2ba5-7640-40fd-964c-ad50264d90fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:27.364076 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1470/2935] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:18:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:18:27.375549 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940ed09-3914-41b7-885d-507d7c6d1a34 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:27.377223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-0940ed09-3914-41b7-885d-507d7c6d1a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:27.398688 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:27.433874 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940ed09-3914-41b7-885d-507d7c6d1a34 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:27.437520 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc44b9d-b6ff-40ef-8b28-3a181c29a850 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:27.438119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc44b9d-b6ff-40ef-8b28-3a181c29a850 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:27.438392 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc44b9d-b6ff-40ef-8b28-3a181c29a850 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:27.438661 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc44b9d-b6ff-40ef-8b28-3a181c29a850 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:27.446836 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc44b9d-b6ff-40ef-8b28-3a181c29a850 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:27.448266 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc44b9d-b6ff-40ef-8b28-3a181c29a850 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:27.457850 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1466/2936] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:18:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:18:31.410522 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc2ba5-7640-40fd-964c-ad50264d90fc None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:31.430000 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:31.472390 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc2ba5-7640-40fd-964c-ad50264d90fc None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:31.477320 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d4aad7-ecb8-4f51-9afe-ab5b114fa821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:31.478009 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4aad7-ecb8-4f51-9afe-ab5b114fa821 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:31.478885 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4aad7-ecb8-4f51-9afe-ab5b114fa821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:31.479155 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4aad7-ecb8-4f51-9afe-ab5b114fa821 admin admin] PATH_INFO: `/v3/users/7631222e250742dd847275be057e85b2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:31.480256 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d4aad7-ecb8-4f51-9afe-ab5b114fa821 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7631222e250742dd847275be057e85b2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:31.482145 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d4aad7-ecb8-4f51-9afe-ab5b114fa821 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:31.645600 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-35d4aad7-ecb8-4f51-9afe-ab5b114fa821 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7631222e250742dd847275be057e85b2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:31.653124 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-35d4aad7-ecb8-4f51-9afe-ab5b114fa821 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7631222e250742dd847275be057e85b2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:31.716606 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-35d4aad7-ecb8-4f51-9afe-ab5b114fa821 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7631222e250742dd847275be057e85b2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:31.717223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-35d4aad7-ecb8-4f51-9afe-ab5b114fa821 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7631222e250742dd847275be057e85b2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:31.730226 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-35d4aad7-ecb8-4f51-9afe-ab5b114fa821 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7631222e250742dd847275be057e85b2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:31.736254 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1471/2937] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 10:18:31 2022] DELETE /identity/v3/users/7631222e250742dd847275be057e85b2 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:18:31.740780 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba156195-e410-47e9-b27c-e2580efda016 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:31.741252 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba156195-e410-47e9-b27c-e2580efda016 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:31.741351 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba156195-e410-47e9-b27c-e2580efda016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:31.760405 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-ba156195-e410-47e9-b27c-e2580efda016 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:18:31.761604 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ba156195-e410-47e9-b27c-e2580efda016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:31.819383 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1467/2938] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:18:31.829440 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d4aad7-ecb8-4f51-9afe-ab5b114fa821 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:31.830914 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-35d4aad7-ecb8-4f51-9afe-ab5b114fa821 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:31.888406 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d4aad7-ecb8-4f51-9afe-ab5b114fa821 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:31.892093 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19401b2c-fb40-4680-bf9b-6eaa4a49c527 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:31.892696 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19401b2c-fb40-4680-bf9b-6eaa4a49c527 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:31.892979 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19401b2c-fb40-4680-bf9b-6eaa4a49c527 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:31.893260 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19401b2c-fb40-4680-bf9b-6eaa4a49c527 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:31.902888 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19401b2c-fb40-4680-bf9b-6eaa4a49c527 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:31.904260 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19401b2c-fb40-4680-bf9b-6eaa4a49c527 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:31.913716 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1472/2939] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:18:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:18:32.295571 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba156195-e410-47e9-b27c-e2580efda016 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:32.296986 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-ba156195-e410-47e9-b27c-e2580efda016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:32.309776 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba156195-e410-47e9-b27c-e2580efda016 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:32.314336 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d091e-3773-44da-91c5-f2b7ad4ddd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:32.315065 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d091e-3773-44da-91c5-f2b7ad4ddd6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:32.315453 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d091e-3773-44da-91c5-f2b7ad4ddd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:32.315650 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d091e-3773-44da-91c5-f2b7ad4ddd6c admin admin] PATH_INFO: `/v3/projects/b4162cb064794fc295aba1c0d39c18c4` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:32.316878 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d091e-3773-44da-91c5-f2b7ad4ddd6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4162cb064794fc295aba1c0d39c18c4)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:32.318817 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d091e-3773-44da-91c5-f2b7ad4ddd6c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:32.426579 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d50d091e-3773-44da-91c5-f2b7ad4ddd6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4162cb064794fc295aba1c0d39c18c4', 'request_id': 'req-d50d091e-3773-44da-91c5-f2b7ad4ddd6c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:32.432730 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d50d091e-3773-44da-91c5-f2b7ad4ddd6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4162cb064794fc295aba1c0d39c18c4', 'request_id': 'req-d50d091e-3773-44da-91c5-f2b7ad4ddd6c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:32.439099 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d50d091e-3773-44da-91c5-f2b7ad4ddd6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4162cb064794fc295aba1c0d39c18c4', 'request_id': 'req-d50d091e-3773-44da-91c5-f2b7ad4ddd6c'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:32.527643 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d50d091e-3773-44da-91c5-f2b7ad4ddd6c admin admin] The token cache is being invalidate because project b4162cb064794fc295aba1c0d39c18c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 10:18:32.527929 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d50d091e-3773-44da-91c5-f2b7ad4ddd6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:32.528572 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-d50d091e-3773-44da-91c5-f2b7ad4ddd6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:18:32.530604 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1468/2940] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 10:18:32 2022] DELETE /identity/v3/projects/b4162cb064794fc295aba1c0d39c18c4 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 10:18:32.535683 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3995a7-4ced-424f-8f33-f93e312add2c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:32.536137 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3995a7-4ced-424f-8f33-f93e312add2c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:32.536227 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3995a7-4ced-424f-8f33-f93e312add2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:32.556268 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ef3995a7-4ced-424f-8f33-f93e312add2c None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:18:32.558103 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ef3995a7-4ced-424f-8f33-f93e312add2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:32.589969 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:32.615165 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1473/2941] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:18:32.623132 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2a005-1051-4370-ae5b-2b99d912ae57 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:32.623421 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2a005-1051-4370-ae5b-2b99d912ae57 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:32.623690 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd2a005-1051-4370-ae5b-2b99d912ae57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:32.643435 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-fbd2a005-1051-4370-ae5b-2b99d912ae57 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:18:32.644681 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fbd2a005-1051-4370-ae5b-2b99d912ae57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:32.652684 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1469/2942] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:18:32.657318 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3995a7-4ced-424f-8f33-f93e312add2c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:32.666734 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3995a7-4ced-424f-8f33-f93e312add2c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:32.671336 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ddafb7-cfcd-4538-b036-34b8a4e7cd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:32.671922 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ddafb7-cfcd-4538-b036-34b8a4e7cd63 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:32.672200 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ddafb7-cfcd-4538-b036-34b8a4e7cd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:32.672471 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ddafb7-cfcd-4538-b036-34b8a4e7cd63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:32.673104 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ddafb7-cfcd-4538-b036-34b8a4e7cd63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 10:18:32.673398 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ddafb7-cfcd-4538-b036-34b8a4e7cd63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:32.675001 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ddafb7-cfcd-4538-b036-34b8a4e7cd63 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:32.687267 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1474/2943] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 10:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 10:18:32.691557 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd2a005-1051-4370-ae5b-2b99d912ae57 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:32.700775 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd2a005-1051-4370-ae5b-2b99d912ae57 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:32.705924 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ad4d1-6b1f-45e4-be33-401340bb704d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:32.706630 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ad4d1-6b1f-45e4-be33-401340bb704d admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:32.706952 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ad4d1-6b1f-45e4-be33-401340bb704d admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:32.707206 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ad4d1-6b1f-45e4-be33-401340bb704d admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:32.707829 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ad4d1-6b1f-45e4-be33-401340bb704d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:32.709697 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ad4d1-6b1f-45e4-be33-401340bb704d admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:32.774768 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1470/2944] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 10:18:32 2022] POST /identity/v3/projects => generated 363 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:18:32.779045 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ddafb7-cfcd-4538-b036-34b8a4e7cd63 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:32.788876 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ddafb7-cfcd-4538-b036-34b8a4e7cd63 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:32.846543 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0436bc9-a528-4c1f-a50e-25db9b889889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:32.846543 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0436bc9-a528-4c1f-a50e-25db9b889889 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:32.846849 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0436bc9-a528-4c1f-a50e-25db9b889889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:32.847376 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0436bc9-a528-4c1f-a50e-25db9b889889 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:32.848013 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0436bc9-a528-4c1f-a50e-25db9b889889 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:32.849938 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0436bc9-a528-4c1f-a50e-25db9b889889 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:32.940295 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1475/2945] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 10:18:32 2022] POST /identity/v3/users => generated 282 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:18:32.944255 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ad4d1-6b1f-45e4-be33-401340bb704d admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:32.954694 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ad4d1-6b1f-45e4-be33-401340bb704d admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:32.959253 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1409a-53ba-46be-ad02-dac79b015f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:32.959852 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1409a-53ba-46be-ad02-dac79b015f75 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:32.960129 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1409a-53ba-46be-ad02-dac79b015f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:32.960397 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1409a-53ba-46be-ad02-dac79b015f75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:32.961094 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1409a-53ba-46be-ad02-dac79b015f75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:32.962664 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1409a-53ba-46be-ad02-dac79b015f75 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:32.972740 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1471/2946] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:18:32.976506 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0436bc9-a528-4c1f-a50e-25db9b889889 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:32.986606 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0436bc9-a528-4c1f-a50e-25db9b889889 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:32.996401 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af5d8bd-7eea-4e71-add0-99e06c617073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:32.997032 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5d8bd-7eea-4e71-add0-99e06c617073 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:32.997310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5d8bd-7eea-4e71-add0-99e06c617073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:32.997581 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5d8bd-7eea-4e71-add0-99e06c617073 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:32.998875 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af5d8bd-7eea-4e71-add0-99e06c617073 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:33.003060 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af5d8bd-7eea-4e71-add0-99e06c617073 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:33.013907 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1476/2947] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:18:33.018379 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1409a-53ba-46be-ad02-dac79b015f75 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:33.029184 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1409a-53ba-46be-ad02-dac79b015f75 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:33.034032 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d221b-0a9c-4961-927b-cea5d4e4ac1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:33.034690 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d221b-0a9c-4961-927b-cea5d4e4ac1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:33.035001 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d221b-0a9c-4961-927b-cea5d4e4ac1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:33.035277 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d221b-0a9c-4961-927b-cea5d4e4ac1c admin admin] PATH_INFO: `/v3/projects/d2ce3a4817e04bf48f250e086bb0f530/users/5ce5813be70e4c8596307c1adf74db81/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:33.037500 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:33.050813 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d221b-0a9c-4961-927b-cea5d4e4ac1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2ce3a4817e04bf48f250e086bb0f530, user_id=5ce5813be70e4c8596307c1adf74db81, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:33.053511 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d221b-0a9c-4961-927b-cea5d4e4ac1c admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:33.111319 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1472/2948] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 10:18:33 2022] PUT /identity/v3/projects/d2ce3a4817e04bf48f250e086bb0f530/users/5ce5813be70e4c8596307c1adf74db81/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:18:33.122933 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca9d98-eace-4dee-84a5-266a1e8fd2cf None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:33.123762 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca9d98-eace-4dee-84a5-266a1e8fd2cf None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:33.124069 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca9d98-eace-4dee-84a5-266a1e8fd2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:33.151469 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a4ca9d98-eace-4dee-84a5-266a1e8fd2cf None None] MFA Rules not processed for user `5ce5813be70e4c8596307c1adf74db81`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:18:33.152678 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a4ca9d98-eace-4dee-84a5-266a1e8fd2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:33.216739 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1477/2949] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:18:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:18:33.227857 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d221b-0a9c-4961-927b-cea5d4e4ac1c admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:18:33.229532 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-369d221b-0a9c-4961-927b-cea5d4e4ac1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:18:33.286134 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d221b-0a9c-4961-927b-cea5d4e4ac1c admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:18:33.289735 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e191a0c7-7370-4329-a6e6-f67753ef50f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b6a1f44f6fe94c09b696867871fd5b47', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:18:33.290329 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191a0c7-7370-4329-a6e6-f67753ef50f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:18:33.290608 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191a0c7-7370-4329-a6e6-f67753ef50f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:18:33.290910 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191a0c7-7370-4329-a6e6-f67753ef50f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:18:33.299528 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e191a0c7-7370-4329-a6e6-f67753ef50f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:18:33.300946 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e191a0c7-7370-4329-a6e6-f67753ef50f5 service cinder] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:18:33.309572 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1473/2950] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:18:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:18:43.213861 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:18:43.305612 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:19:22.242511 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca9d98-eace-4dee-84a5-266a1e8fd2cf None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:19:22.321368 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca9d98-eace-4dee-84a5-266a1e8fd2cf None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:19:22.325825 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c16464-c0ca-473f-a4ff-f2fe22cd479e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:19:22.326443 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c16464-c0ca-473f-a4ff-f2fe22cd479e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:19:22.326736 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c16464-c0ca-473f-a4ff-f2fe22cd479e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:19:22.327014 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c16464-c0ca-473f-a4ff-f2fe22cd479e admin admin] PATH_INFO: `/v3/users/5ce5813be70e4c8596307c1adf74db81` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:19:22.328119 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c16464-c0ca-473f-a4ff-f2fe22cd479e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ce5813be70e4c8596307c1adf74db81)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:19:22.329996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c16464-c0ca-473f-a4ff-f2fe22cd479e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:19:22.520538 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-69c16464-c0ca-473f-a4ff-f2fe22cd479e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ce5813be70e4c8596307c1adf74db81'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:19:22.527629 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-69c16464-c0ca-473f-a4ff-f2fe22cd479e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ce5813be70e4c8596307c1adf74db81'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:19:22.601318 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-69c16464-c0ca-473f-a4ff-f2fe22cd479e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ce5813be70e4c8596307c1adf74db81'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:19:22.601939 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-69c16464-c0ca-473f-a4ff-f2fe22cd479e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ce5813be70e4c8596307c1adf74db81'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:19:22.616130 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-69c16464-c0ca-473f-a4ff-f2fe22cd479e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ce5813be70e4c8596307c1adf74db81'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:19:22.624145 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1478/2951] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 10:19:22 2022] DELETE /identity/v3/users/5ce5813be70e4c8596307c1adf74db81 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:19:22.629178 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f4f18-b633-4349-a49e-29805faa1f04 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:19:22.629665 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f4f18-b633-4349-a49e-29805faa1f04 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:19:22.629805 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f4f18-b633-4349-a49e-29805faa1f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:19:22.669108 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f77f4f18-b633-4349-a49e-29805faa1f04 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:19:22.670313 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f77f4f18-b633-4349-a49e-29805faa1f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:19:22.727055 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1474/2952] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:19:22.736669 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c16464-c0ca-473f-a4ff-f2fe22cd479e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:19:22.738061 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-69c16464-c0ca-473f-a4ff-f2fe22cd479e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:19:22.807302 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c16464-c0ca-473f-a4ff-f2fe22cd479e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:19:22.810810 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7904de74-fba0-460b-b833-d74abae0fd72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:19:22.811402 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904de74-fba0-460b-b833-d74abae0fd72 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:19:22.811676 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904de74-fba0-460b-b833-d74abae0fd72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:19:22.811947 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7904de74-fba0-460b-b833-d74abae0fd72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:19:22.817689 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:19:22.821241 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7904de74-fba0-460b-b833-d74abae0fd72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:19:22.822645 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7904de74-fba0-460b-b833-d74abae0fd72 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:19:22.832002 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1479/2953] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:19:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:19:23.345362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f4f18-b633-4349-a49e-29805faa1f04 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:19:23.346648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f77f4f18-b633-4349-a49e-29805faa1f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:19:23.359669 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f4f18-b633-4349-a49e-29805faa1f04 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:19:23.364712 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e5d77-a305-4c20-8bc2-3e9bf81bef88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:19:23.365356 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e5d77-a305-4c20-8bc2-3e9bf81bef88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:19:23.365632 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e5d77-a305-4c20-8bc2-3e9bf81bef88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:19:23.365903 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e5d77-a305-4c20-8bc2-3e9bf81bef88 admin admin] PATH_INFO: `/v3/projects/d2ce3a4817e04bf48f250e086bb0f530` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:19:23.367066 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434e5d77-a305-4c20-8bc2-3e9bf81bef88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2ce3a4817e04bf48f250e086bb0f530)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:19:23.368950 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434e5d77-a305-4c20-8bc2-3e9bf81bef88 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:19:23.504182 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-434e5d77-a305-4c20-8bc2-3e9bf81bef88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2ce3a4817e04bf48f250e086bb0f530', 'request_id': 'req-434e5d77-a305-4c20-8bc2-3e9bf81bef88'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:19:23.510119 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-434e5d77-a305-4c20-8bc2-3e9bf81bef88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2ce3a4817e04bf48f250e086bb0f530', 'request_id': 'req-434e5d77-a305-4c20-8bc2-3e9bf81bef88'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:19:23.516419 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-434e5d77-a305-4c20-8bc2-3e9bf81bef88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2ce3a4817e04bf48f250e086bb0f530', 'request_id': 'req-434e5d77-a305-4c20-8bc2-3e9bf81bef88'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:19:23.618211 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-434e5d77-a305-4c20-8bc2-3e9bf81bef88 admin admin] The token cache is being invalidate because project d2ce3a4817e04bf48f250e086bb0f530 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81611) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 10:19:23.618498 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-434e5d77-a305-4c20-8bc2-3e9bf81bef88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:19:23.619139 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-434e5d77-a305-4c20-8bc2-3e9bf81bef88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:19:23.626842 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1475/2954] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 10:19:23 2022] DELETE /identity/v3/projects/d2ce3a4817e04bf48f250e086bb0f530 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:19:23.626954 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db6dff-15ba-4314-9e63-61b0dc922c19 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:19:23.626954 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db6dff-15ba-4314-9e63-61b0dc922c19 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:19:23.626954 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31db6dff-15ba-4314-9e63-61b0dc922c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:19:23.673252 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-31db6dff-15ba-4314-9e63-61b0dc922c19 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:19:23.674466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-31db6dff-15ba-4314-9e63-61b0dc922c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:19:23.730829 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1480/2955] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:19:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:19:23.738692 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a562d-14cb-4983-ad02-a1082072dcf7 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:19:23.739162 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a562d-14cb-4983-ad02-a1082072dcf7 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:19:23.739925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a562d-14cb-4983-ad02-a1082072dcf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:19:23.759362 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-7c9a562d-14cb-4983-ad02-a1082072dcf7 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:19:23.760574 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-7c9a562d-14cb-4983-ad02-a1082072dcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:19:23.768428 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1476/2956] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:19:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 02 10:19:23.774478 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db6dff-15ba-4314-9e63-61b0dc922c19 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:19:23.784545 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31db6dff-15ba-4314-9e63-61b0dc922c19 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:19:23.789576 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a89ba5-50b5-43b5-8ec4-20b8d410cb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:19:23.790193 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a89ba5-50b5-43b5-8ec4-20b8d410cb81 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:19:23.790494 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a89ba5-50b5-43b5-8ec4-20b8d410cb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:19:23.790790 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a89ba5-50b5-43b5-8ec4-20b8d410cb81 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:19:23.791466 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a89ba5-50b5-43b5-8ec4-20b8d410cb81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 02 10:19:23.791791 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a89ba5-50b5-43b5-8ec4-20b8d410cb81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:19:23.793363 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a89ba5-50b5-43b5-8ec4-20b8d410cb81 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:19:23.805429 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1481/2957] 10.222.0.57 () {58 vars in 1153 bytes} [Fri Sep 2 10:19:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 02 10:19:23.809725 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a562d-14cb-4983-ad02-a1082072dcf7 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:19:23.819228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9a562d-14cb-4983-ad02-a1082072dcf7 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:19:23.823908 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ba61d-aeb8-4278-8f68-78e1f88b63e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:19:23.824505 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ba61d-aeb8-4278-8f68-78e1f88b63e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:19:23.824788 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ba61d-aeb8-4278-8f68-78e1f88b63e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:19:23.825062 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ba61d-aeb8-4278-8f68-78e1f88b63e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:19:23.825689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ba61d-aeb8-4278-8f68-78e1f88b63e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:19:23.827625 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4ba61d-aeb8-4278-8f68-78e1f88b63e2 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:19:23.922011 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1477/2958] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 10:19:23 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:19:23.926071 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a89ba5-50b5-43b5-8ec4-20b8d410cb81 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:19:23.935987 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a89ba5-50b5-43b5-8ec4-20b8d410cb81 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:19:23.990991 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da76ec-8890-44c2-9569-647ffc2e42f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:19:23.991617 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da76ec-8890-44c2-9569-647ffc2e42f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:19:23.991895 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da76ec-8890-44c2-9569-647ffc2e42f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:19:23.992163 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da76ec-8890-44c2-9569-647ffc2e42f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:19:23.992776 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da76ec-8890-44c2-9569-647ffc2e42f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:19:23.994652 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da76ec-8890-44c2-9569-647ffc2e42f7 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:19:24.110057 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1482/2959] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 10:19:23 2022] POST /identity/v3/users => generated 287 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:19:24.114056 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ba61d-aeb8-4278-8f68-78e1f88b63e2 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:19:24.124251 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ba61d-aeb8-4278-8f68-78e1f88b63e2 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:19:24.128836 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585f434-f311-452b-9d24-6488f733a094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:19:24.129431 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585f434-f311-452b-9d24-6488f733a094 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:19:24.129709 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585f434-f311-452b-9d24-6488f733a094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:19:24.129980 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585f434-f311-452b-9d24-6488f733a094 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:19:24.130620 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585f434-f311-452b-9d24-6488f733a094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:19:24.132343 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585f434-f311-452b-9d24-6488f733a094 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:19:24.142272 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1478/2960] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:19:24.146513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da76ec-8890-44c2-9569-647ffc2e42f7 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:19:24.156397 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da76ec-8890-44c2-9569-647ffc2e42f7 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:19:24.160913 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69406c29-f44a-44b4-90a7-72372c451784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:19:24.162117 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69406c29-f44a-44b4-90a7-72372c451784 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:19:24.162395 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69406c29-f44a-44b4-90a7-72372c451784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:19:24.162664 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69406c29-f44a-44b4-90a7-72372c451784 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:19:24.163327 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69406c29-f44a-44b4-90a7-72372c451784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:19:24.164880 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69406c29-f44a-44b4-90a7-72372c451784 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:19:24.175368 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1483/2961] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:19:24.179310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585f434-f311-452b-9d24-6488f733a094 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:19:24.189180 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585f434-f311-452b-9d24-6488f733a094 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:19:24.193762 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bafc98-cb3a-4036-a3f0-4185f2610cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:19:24.194440 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bafc98-cb3a-4036-a3f0-4185f2610cae admin admin] REQUEST_METHOD: `PUT` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:19:24.194751 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bafc98-cb3a-4036-a3f0-4185f2610cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:19:24.195034 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bafc98-cb3a-4036-a3f0-4185f2610cae admin admin] PATH_INFO: `/v3/projects/bfd00fdbe14e451fb3634ef9077b3a76/users/a29bbe80a3cc44a58ddcd0dd784f11bf/roles/efe6bdd5112340b48b262ceafbc19c04` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:19:24.210024 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bafc98-cb3a-4036-a3f0-4185f2610cae admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfd00fdbe14e451fb3634ef9077b3a76, user_id=a29bbe80a3cc44a58ddcd0dd784f11bf, role_id=efe6bdd5112340b48b262ceafbc19c04)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:19:24.212660 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bafc98-cb3a-4036-a3f0-4185f2610cae admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:19:24.214835 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:19:24.309795 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1479/2962] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 10:19:24 2022] PUT /identity/v3/projects/bfd00fdbe14e451fb3634ef9077b3a76/users/a29bbe80a3cc44a58ddcd0dd784f11bf/roles/efe6bdd5112340b48b262ceafbc19c04 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:19:24.314833 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1f4b0-fc0c-474d-9f35-3bdfa257d547 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:19:24.315121 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1f4b0-fc0c-474d-9f35-3bdfa257d547 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:19:24.315399 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1f4b0-fc0c-474d-9f35-3bdfa257d547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:19:24.333778 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-45f1f4b0-fc0c-474d-9f35-3bdfa257d547 None None] MFA Rules not processed for user `9cffe229f2f843e394c2248b000ecb37`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:19:24.335013 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-45f1f4b0-fc0c-474d-9f35-3bdfa257d547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:19:24.365460 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:19:24.393026 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1484/2963] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:19:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:19:24.402532 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bafc98-cb3a-4036-a3f0-4185f2610cae admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:19:24.404277 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-a0bafc98-cb3a-4036-a3f0-4185f2610cae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:19:24.461689 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bafc98-cb3a-4036-a3f0-4185f2610cae admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:19:24.465392 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae740e0a-3f06-401b-ad4c-42271884b191 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:19:24.465992 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae740e0a-3f06-401b-ad4c-42271884b191 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:19:24.466269 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae740e0a-3f06-401b-ad4c-42271884b191 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:19:24.466541 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae740e0a-3f06-401b-ad4c-42271884b191 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:19:24.475713 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae740e0a-3f06-401b-ad4c-42271884b191 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:19:24.477121 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae740e0a-3f06-401b-ad4c-42271884b191 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:19:24.486247 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1480/2964] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:19:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:19:29.813855 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c2e9a-c3de-4523-92a1-f757cc69bd1b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:19:29.814350 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c2e9a-c3de-4523-92a1-f757cc69bd1b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:19:29.814463 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c2e9a-c3de-4523-92a1-f757cc69bd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:19:29.847206 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-923c2e9a-c3de-4523-92a1-f757cc69bd1b None None] MFA Rules not processed for user `a29bbe80a3cc44a58ddcd0dd784f11bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:19:29.848203 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-923c2e9a-c3de-4523-92a1-f757cc69bd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:19:29.918799 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1485/2965] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:19:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:19:29.923289 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae740e0a-3f06-401b-ad4c-42271884b191 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:19:29.933954 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae740e0a-3f06-401b-ad4c-42271884b191 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:19:29.938949 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c91bb-1bf2-4826-88d4-3833804fdb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:19:29.945367 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c91bb-1bf2-4826-88d4-3833804fdb15 admin admin] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:19:29.945367 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c91bb-1bf2-4826-88d4-3833804fdb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:19:29.945367 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c91bb-1bf2-4826-88d4-3833804fdb15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:19:29.945367 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c91bb-1bf2-4826-88d4-3833804fdb15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:19:29.945367 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c91bb-1bf2-4826-88d4-3833804fdb15 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:19:30.025660 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1481/2966] 10.222.0.57 () {60 vars in 1153 bytes} [Fri Sep 2 10:19:29 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:19:30.029926 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c2e9a-c3de-4523-92a1-f757cc69bd1b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:19:30.040797 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c2e9a-c3de-4523-92a1-f757cc69bd1b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:19:30.097380 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57cfa0-0541-4708-abcd-44af8e7a040a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:19:30.097977 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57cfa0-0541-4708-abcd-44af8e7a040a admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:19:30.098251 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57cfa0-0541-4708-abcd-44af8e7a040a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:19:30.098514 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc57cfa0-0541-4708-abcd-44af8e7a040a admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:19:30.099160 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57cfa0-0541-4708-abcd-44af8e7a040a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:19:30.101048 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc57cfa0-0541-4708-abcd-44af8e7a040a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:19:30.155279 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1486/2967] 10.222.0.57 () {60 vars in 1144 bytes} [Fri Sep 2 10:19:30 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 02 10:19:30.159393 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c91bb-1bf2-4826-88d4-3833804fdb15 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:19:30.170407 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c91bb-1bf2-4826-88d4-3833804fdb15 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:19:30.175366 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0582db7a-f945-498c-a25c-9b7b9134abe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:19:30.175985 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582db7a-f945-498c-a25c-9b7b9134abe3 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:19:30.176296 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582db7a-f945-498c-a25c-9b7b9134abe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:19:30.177268 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582db7a-f945-498c-a25c-9b7b9134abe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:19:30.177951 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0582db7a-f945-498c-a25c-9b7b9134abe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:19:30.179639 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0582db7a-f945-498c-a25c-9b7b9134abe3 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:19:30.190842 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1482/2968] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 02 10:19:30.195783 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57cfa0-0541-4708-abcd-44af8e7a040a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:19:30.205846 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc57cfa0-0541-4708-abcd-44af8e7a040a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:19:30.210412 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b9179-4fae-4f92-b85c-7a52a09d641f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:19:30.211100 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b9179-4fae-4f92-b85c-7a52a09d641f admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:19:30.211381 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b9179-4fae-4f92-b85c-7a52a09d641f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:19:30.211654 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3b9179-4fae-4f92-b85c-7a52a09d641f admin admin] PATH_INFO: `/v3/projects/1a25144970a84b8eb90df62ab2232985/users/5fa378fcb82c492aa01013b3a2c7edbd/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:19:30.226068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b9179-4fae-4f92-b85c-7a52a09d641f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a25144970a84b8eb90df62ab2232985, user_id=5fa378fcb82c492aa01013b3a2c7edbd, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:19:30.228722 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3b9179-4fae-4f92-b85c-7a52a09d641f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:19:30.292156 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1487/2969] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 10:19:30 2022] PUT /identity/v3/projects/1a25144970a84b8eb90df62ab2232985/users/5fa378fcb82c492aa01013b3a2c7edbd/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 02 10:19:30.295964 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0582db7a-f945-498c-a25c-9b7b9134abe3 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:19:30.306800 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0582db7a-f945-498c-a25c-9b7b9134abe3 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:19:30.366377 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c8cfc-b6db-4a05-af4c-ad919e1fd299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:19:30.367042 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c8cfc-b6db-4a05-af4c-ad919e1fd299 admin admin] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:19:30.367325 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c8cfc-b6db-4a05-af4c-ad919e1fd299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:19:30.367599 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c8cfc-b6db-4a05-af4c-ad919e1fd299 admin admin] PATH_INFO: `/v3/roles` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:19:30.368274 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5c8cfc-b6db-4a05-af4c-ad919e1fd299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:19:30.369876 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5c8cfc-b6db-4a05-af4c-ad919e1fd299 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:19:30.379702 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1483/2970] 10.222.0.57 () {58 vars in 1122 bytes} [Fri Sep 2 10:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 02 10:19:30.383853 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b9179-4fae-4f92-b85c-7a52a09d641f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:19:30.394787 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3b9179-4fae-4f92-b85c-7a52a09d641f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:19:30.399699 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9c5c5-d7e0-4361-8867-e54b51609b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:19:30.400360 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9c5c5-d7e0-4361-8867-e54b51609b2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:19:30.400645 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9c5c5-d7e0-4361-8867-e54b51609b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:19:30.400924 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b9c5c5-d7e0-4361-8867-e54b51609b2d admin admin] PATH_INFO: `/v3/projects/1a25144970a84b8eb90df62ab2232985/users/5fa378fcb82c492aa01013b3a2c7edbd/roles/f886b8e8889549e896a8fb3426cce9e1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:19:30.403040 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9c5c5-d7e0-4361-8867-e54b51609b2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a25144970a84b8eb90df62ab2232985, user_id=5fa378fcb82c492aa01013b3a2c7edbd, role_id=f886b8e8889549e896a8fb3426cce9e1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:19:30.405543 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b9c5c5-d7e0-4361-8867-e54b51609b2d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:19:30.417231 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1488/2971] 10.222.0.57 () {60 vars in 1483 bytes} [Fri Sep 2 10:19:30 2022] PUT /identity/v3/projects/1a25144970a84b8eb90df62ab2232985/users/5fa378fcb82c492aa01013b3a2c7edbd/roles/f886b8e8889549e896a8fb3426cce9e1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:19:35.908593 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081aabd-30dc-4734-b945-c3040659dcae None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:19:35.909079 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081aabd-30dc-4734-b945-c3040659dcae None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:19:35.909183 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081aabd-30dc-4734-b945-c3040659dcae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:19:35.937087 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d081aabd-30dc-4734-b945-c3040659dcae None None] MFA Rules not processed for user `5fa378fcb82c492aa01013b3a2c7edbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:19:35.938217 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d081aabd-30dc-4734-b945-c3040659dcae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:19:35.987197 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:19:36.008699 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1484/2972] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:19:35 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:19:36.025321 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9c5c5-d7e0-4361-8867-e54b51609b2d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:19:36.027096 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-72b9c5c5-d7e0-4361-8867-e54b51609b2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:19:36.048414 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:19:36.103560 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b9c5c5-d7e0-4361-8867-e54b51609b2d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:19:36.109522 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb174d34-1aec-4ad2-8ac6-7f90c73951b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:19:36.110131 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb174d34-1aec-4ad2-8ac6-7f90c73951b7 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:19:36.110414 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb174d34-1aec-4ad2-8ac6-7f90c73951b7 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:19:36.110784 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb174d34-1aec-4ad2-8ac6-7f90c73951b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:19:36.166764 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb174d34-1aec-4ad2-8ac6-7f90c73951b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:19:36.168308 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb174d34-1aec-4ad2-8ac6-7f90c73951b7 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:19:36.177681 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1489/2973] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:19:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:19:38.651223 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08561c2-d50e-4ba4-bd05-e3a634d246a2 None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:19:38.651809 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08561c2-d50e-4ba4-bd05-e3a634d246a2 None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:19:38.651809 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08561c2-d50e-4ba4-bd05-e3a634d246a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:19:38.675861 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-f08561c2-d50e-4ba4-bd05-e3a634d246a2 None None] MFA Rules not processed for user `a29bbe80a3cc44a58ddcd0dd784f11bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:19:38.677137 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f08561c2-d50e-4ba4-bd05-e3a634d246a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:19:38.684590 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1485/2974] 10.222.0.57 () {58 vars in 958 bytes} [Fri Sep 2 10:19:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:19:46.173151 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=48 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:19:48.668434 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=10 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:20:06.484375 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb174d34-1aec-4ad2-8ac6-7f90c73951b7 service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:20:06.496952 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb174d34-1aec-4ad2-8ac6-7f90c73951b7 service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:20:06.502645 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b3b28-2719-4ca4-9ac3-1dc9f07092cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:20:06.504758 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b3b28-2719-4ca4-9ac3-1dc9f07092cf service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:20:06.504758 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b3b28-2719-4ca4-9ac3-1dc9f07092cf service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:20:06.504758 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699b3b28-2719-4ca4-9ac3-1dc9f07092cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:20:06.505570 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-699b3b28-2719-4ca4-9ac3-1dc9f07092cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:20:06.532130 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699b3b28-2719-4ca4-9ac3-1dc9f07092cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:20:06.533715 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699b3b28-2719-4ca4-9ac3-1dc9f07092cf service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:20:06.544835 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1490/2975] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:20:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:20:16.540285 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=11 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:20:18.455338 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08561c2-d50e-4ba4-bd05-e3a634d246a2 None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:20:18.468542 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08561c2-d50e-4ba4-bd05-e3a634d246a2 None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:20:18.474871 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae8df4-141d-4f44-b79e-22991c0a4af7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1da355c8be2d496bb08feb0fe74e04b9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:20:18.475521 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae8df4-141d-4f44-b79e-22991c0a4af7 service nova] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:20:18.475884 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae8df4-141d-4f44-b79e-22991c0a4af7 service nova] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:20:18.476161 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae8df4-141d-4f44-b79e-22991c0a4af7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:20:18.477897 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-28ae8df4-141d-4f44-b79e-22991c0a4af7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:20:18.490941 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae8df4-141d-4f44-b79e-22991c0a4af7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:20:18.492442 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae8df4-141d-4f44-b79e-22991c0a4af7 service nova] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:20:18.502992 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1486/2976] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:20:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:20:18.916298 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b3b28-2719-4ca4-9ac3-1dc9f07092cf service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:20:18.971037 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699b3b28-2719-4ca4-9ac3-1dc9f07092cf service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:20:18.975329 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ff83f-cc38-49d8-8f3b-2b44f0f390bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:20:18.975931 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ff83f-cc38-49d8-8f3b-2b44f0f390bf service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:20:18.976210 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ff83f-cc38-49d8-8f3b-2b44f0f390bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:20:18.976486 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ff83f-cc38-49d8-8f3b-2b44f0f390bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:20:18.978220 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-679ff83f-cc38-49d8-8f3b-2b44f0f390bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:20:18.987914 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ff83f-cc38-49d8-8f3b-2b44f0f390bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:20:18.989520 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ff83f-cc38-49d8-8f3b-2b44f0f390bf service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:20:18.997901 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1491/2977] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:20:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:20:21.693277 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae8df4-141d-4f44-b79e-22991c0a4af7 service nova] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:20:21.694724 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-28ae8df4-141d-4f44-b79e-22991c0a4af7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:20:21.705994 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae8df4-141d-4f44-b79e-22991c0a4af7 service nova] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:20:21.709535 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae2660-41a9-4402-96e4-58b62948bc2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:20:21.710616 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae2660-41a9-4402-96e4-58b62948bc2c service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:20:21.710921 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae2660-41a9-4402-96e4-58b62948bc2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:20:21.711190 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae2660-41a9-4402-96e4-58b62948bc2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:20:21.720165 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ae2660-41a9-4402-96e4-58b62948bc2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:20:21.721560 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ae2660-41a9-4402-96e4-58b62948bc2c service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:20:21.731357 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1487/2978] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:20:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:20:28.994269 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=26 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:20:31.727314 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=12 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:22:16.090671 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ff83f-cc38-49d8-8f3b-2b44f0f390bf service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:22:16.092122 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-679ff83f-cc38-49d8-8f3b-2b44f0f390bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:22:16.197648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ff83f-cc38-49d8-8f3b-2b44f0f390bf service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:22:16.202511 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a5f7b-d6b8-425a-a3c8-8b8094d6de76 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5177fa4d168c4c219e09c7178bbbc790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:22:16.203154 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a5f7b-d6b8-425a-a3c8-8b8094d6de76 service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:22:16.203434 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a5f7b-d6b8-425a-a3c8-8b8094d6de76 service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:22:16.203704 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a5f7b-d6b8-425a-a3c8-8b8094d6de76 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:22:16.205403 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ba7a5f7b-d6b8-425a-a3c8-8b8094d6de76 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:22:16.217461 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a5f7b-d6b8-425a-a3c8-8b8094d6de76 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:22:16.219301 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a5f7b-d6b8-425a-a3c8-8b8094d6de76 service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:22:16.230086 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1492/2979] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:22:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:22:26.225639 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=48 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:22:37.973273 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f6e27-36cd-4afe-9633-05c00208c21a None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:22:37.973852 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f6e27-36cd-4afe-9633-05c00208c21a None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:22:37.973852 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f6e27-36cd-4afe-9633-05c00208c21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:22:38.027604 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-038f6e27-36cd-4afe-9633-05c00208c21a None None] MFA Rules not processed for user `dfea2ee0c9e84b48ac9bafe3173d937a`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:22:38.028893 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-038f6e27-36cd-4afe-9633-05c00208c21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:22:38.039046 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1488/2980] 10.222.0.57 () {58 vars in 1062 bytes} [Fri Sep 2 10:22:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:22:38.046012 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a5f7b-d6b8-425a-a3c8-8b8094d6de76 service placement] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:22:38.056295 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a5f7b-d6b8-425a-a3c8-8b8094d6de76 service placement] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:22:38.059864 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae498e7-d50b-475b-a91d-15caaaf5f269 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:22:38.060484 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae498e7-d50b-475b-a91d-15caaaf5f269 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:22:38.060764 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae498e7-d50b-475b-a91d-15caaaf5f269 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:22:38.061038 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae498e7-d50b-475b-a91d-15caaaf5f269 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:22:38.062817 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bae498e7-d50b-475b-a91d-15caaaf5f269 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:22:38.076978 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae498e7-d50b-475b-a91d-15caaaf5f269 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:22:38.078724 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae498e7-d50b-475b-a91d-15caaaf5f269 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:22:38.091641 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1493/2981] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:22:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:22:39.282768 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe700c74-ab9f-4cc7-a943-8283b55273ae None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:22:39.283294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe700c74-ab9f-4cc7-a943-8283b55273ae None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:22:39.283408 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe700c74-ab9f-4cc7-a943-8283b55273ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:22:39.303345 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-fe700c74-ab9f-4cc7-a943-8283b55273ae None None] MFA Rules not processed for user `dfea2ee0c9e84b48ac9bafe3173d937a`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:22:39.304603 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-fe700c74-ab9f-4cc7-a943-8283b55273ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:22:39.311633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1489/2982] 10.222.0.57 () {58 vars in 1062 bytes} [Fri Sep 2 10:22:39 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:22:39.317587 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae498e7-d50b-475b-a91d-15caaaf5f269 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:22:39.326447 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae498e7-d50b-475b-a91d-15caaaf5f269 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:22:39.330034 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab8cbb-e38a-4f8e-833f-5a88d9962a9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:22:39.330625 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab8cbb-e38a-4f8e-833f-5a88d9962a9f service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:22:39.330946 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab8cbb-e38a-4f8e-833f-5a88d9962a9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:22:39.331223 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab8cbb-e38a-4f8e-833f-5a88d9962a9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:22:39.332951 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-efab8cbb-e38a-4f8e-833f-5a88d9962a9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:22:39.353575 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab8cbb-e38a-4f8e-833f-5a88d9962a9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:22:39.355032 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab8cbb-e38a-4f8e-833f-5a88d9962a9f service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:22:39.365808 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1494/2983] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:22:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:22:49.298554 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=16 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:22:49.360941 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=12 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:22:52.435218 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe700c74-ab9f-4cc7-a943-8283b55273ae None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:22:52.443221 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe700c74-ab9f-4cc7-a943-8283b55273ae None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:22:52.447335 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878fd8f5-e4a7-4f9d-8461-3d99191bbf2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:22:52.448006 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878fd8f5-e4a7-4f9d-8461-3d99191bbf2f service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:22:52.448289 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878fd8f5-e4a7-4f9d-8461-3d99191bbf2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:22:52.448581 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878fd8f5-e4a7-4f9d-8461-3d99191bbf2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:22:52.450363 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-878fd8f5-e4a7-4f9d-8461-3d99191bbf2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:22:52.460985 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878fd8f5-e4a7-4f9d-8461-3d99191bbf2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:22:52.465411 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878fd8f5-e4a7-4f9d-8461-3d99191bbf2f service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:22:52.471461 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1490/2984] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:22:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:22:53.012790 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc37a9-c0cd-40f3-ac01-0cc7c65ab0c2 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:22:53.013276 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc37a9-c0cd-40f3-ac01-0cc7c65ab0c2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:22:53.013513 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc37a9-c0cd-40f3-ac01-0cc7c65ab0c2 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:22:53.014646 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1495/2985] 10.222.0.57 () {54 vars in 921 bytes} [Fri Sep 2 10:22:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 10:22:53.020590 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205d886-c0dd-4a48-a476-e96fe7e9e69f None None] REQUEST_METHOD: `POST` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:22:53.020910 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205d886-c0dd-4a48-a476-e96fe7e9e69f None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:22:53.021177 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205d886-c0dd-4a48-a476-e96fe7e9e69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:22:53.040925 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.auth.core [None req-d205d886-c0dd-4a48-a476-e96fe7e9e69f None None] MFA Rules not processed for user `dfea2ee0c9e84b48ac9bafe3173d937a`. Rule list: `[]` (Enabled: `True`). {{(pid=81611) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:22:53.042137 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-d205d886-c0dd-4a48-a476-e96fe7e9e69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:22:53.049721 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1491/2986] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Sep 2 10:22:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:22:53.061647 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc37a9-c0cd-40f3-ac01-0cc7c65ab0c2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:22:53.075709 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc37a9-c0cd-40f3-ac01-0cc7c65ab0c2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:22:53.079805 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cab698-4a2b-49d8-ab7b-8c371cfd684f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:22:53.080400 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cab698-4a2b-49d8-ab7b-8c371cfd684f service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:22:53.080683 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cab698-4a2b-49d8-ab7b-8c371cfd684f service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:22:53.080955 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cab698-4a2b-49d8-ab7b-8c371cfd684f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:22:53.089778 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cab698-4a2b-49d8-ab7b-8c371cfd684f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:22:53.091221 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cab698-4a2b-49d8-ab7b-8c371cfd684f service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:22:53.099684 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1496/2987] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:22:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:22:53.182848 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0fbb7f-dbc8-4443-93f8-af746b4c9dda None None] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:22:53.183136 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0fbb7f-dbc8-4443-93f8-af746b4c9dda None None] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:22:53.183409 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0fbb7f-dbc8-4443-93f8-af746b4c9dda None None] PATH_INFO: `/` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:22:53.184767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1492/2988] 10.222.0.57 () {54 vars in 921 bytes} [Fri Sep 2 10:22:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 02 10:22:53.190052 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b4a9c-c5bf-42fa-a75e-bb1fcc18aae9 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:22:53.190339 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b4a9c-c5bf-42fa-a75e-bb1fcc18aae9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:22:53.190614 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b4a9c-c5bf-42fa-a75e-bb1fcc18aae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:22:53.230310 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ae5b4a9c-c5bf-42fa-a75e-bb1fcc18aae9 None None] MFA Rules not processed for user `1da355c8be2d496bb08feb0fe74e04b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:22:53.231697 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ae5b4a9c-c5bf-42fa-a75e-bb1fcc18aae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:22:53.240996 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1497/2989] 10.222.0.57 () {58 vars in 1018 bytes} [Fri Sep 2 10:22:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:22:53.251638 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0fbb7f-dbc8-4443-93f8-af746b4c9dda None None] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:22:53.262446 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0fbb7f-dbc8-4443-93f8-af746b4c9dda None None] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:22:53.266556 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764e014-a03b-4018-98c9-cf0b43d9630f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:22:53.267212 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764e014-a03b-4018-98c9-cf0b43d9630f service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:22:53.267493 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764e014-a03b-4018-98c9-cf0b43d9630f service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:22:53.267767 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4764e014-a03b-4018-98c9-cf0b43d9630f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:22:53.277771 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764e014-a03b-4018-98c9-cf0b43d9630f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:22:53.279227 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4764e014-a03b-4018-98c9-cf0b43d9630f service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:22:53.290931 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1493/2990] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:22:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:22:53.351830 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b4a9c-c5bf-42fa-a75e-bb1fcc18aae9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:22:53.353200 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ae5b4a9c-c5bf-42fa-a75e-bb1fcc18aae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:22:53.368517 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b4a9c-c5bf-42fa-a75e-bb1fcc18aae9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:22:53.373619 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584edf5a-6cd4-43b5-8d1f-7ecae3387105 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5177fa4d168c4c219e09c7178bbbc790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:22:53.374219 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584edf5a-6cd4-43b5-8d1f-7ecae3387105 service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:22:53.374495 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584edf5a-6cd4-43b5-8d1f-7ecae3387105 service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:22:53.374803 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584edf5a-6cd4-43b5-8d1f-7ecae3387105 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:22:53.376498 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-584edf5a-6cd4-43b5-8d1f-7ecae3387105 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:22:53.390471 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584edf5a-6cd4-43b5-8d1f-7ecae3387105 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:22:53.392564 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584edf5a-6cd4-43b5-8d1f-7ecae3387105 service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:22:53.402258 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1498/2991] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:22:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:23:03.286052 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=15 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:23:03.398039 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=21 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:24:39.545860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764e014-a03b-4018-98c9-cf0b43d9630f service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:24:39.554310 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4764e014-a03b-4018-98c9-cf0b43d9630f service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:24:39.557971 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34672d6-bb92-4012-8eed-f8bebc07e652 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:24:39.558568 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34672d6-bb92-4012-8eed-f8bebc07e652 service neutron] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:24:39.558878 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34672d6-bb92-4012-8eed-f8bebc07e652 service neutron] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:24:39.559149 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34672d6-bb92-4012-8eed-f8bebc07e652 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:24:39.567811 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34672d6-bb92-4012-8eed-f8bebc07e652 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:24:39.569232 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34672d6-bb92-4012-8eed-f8bebc07e652 service neutron] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:24:39.577992 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1494/2992] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:24:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:24:45.874475 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584edf5a-6cd4-43b5-8d1f-7ecae3387105 service placement] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:24:45.884800 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584edf5a-6cd4-43b5-8d1f-7ecae3387105 service placement] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:24:45.888328 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a0c58-ff37-4d20-bb04-3abaa29ef2ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dfea2ee0c9e84b48ac9bafe3173d937a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:24:45.888917 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a0c58-ff37-4d20-bb04-3abaa29ef2ff service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:24:45.889189 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a0c58-ff37-4d20-bb04-3abaa29ef2ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:24:45.889454 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a0c58-ff37-4d20-bb04-3abaa29ef2ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:24:45.950068 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173a0c58-ff37-4d20-bb04-3abaa29ef2ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:24:45.951692 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173a0c58-ff37-4d20-bb04-3abaa29ef2ff service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:24:45.961367 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1499/2993] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:24:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:24:49.574301 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:24:49.905680 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34672d6-bb92-4012-8eed-f8bebc07e652 service neutron] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:24:49.925768 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34672d6-bb92-4012-8eed-f8bebc07e652 service neutron] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:24:49.931474 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a93b1-ff07-4466-a2aa-796d0474c10a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5177fa4d168c4c219e09c7178bbbc790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:24:49.932098 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a93b1-ff07-4466-a2aa-796d0474c10a service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:24:49.932380 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a93b1-ff07-4466-a2aa-796d0474c10a service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:24:49.932648 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a93b1-ff07-4466-a2aa-796d0474c10a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:24:49.934470 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-741a93b1-ff07-4466-a2aa-796d0474c10a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:24:49.947532 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741a93b1-ff07-4466-a2aa-796d0474c10a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:24:49.949108 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741a93b1-ff07-4466-a2aa-796d0474c10a service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:24:49.960945 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1495/2994] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:24:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:24:53.587912 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a0c58-ff37-4d20-bb04-3abaa29ef2ff service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:24:53.599606 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a0c58-ff37-4d20-bb04-3abaa29ef2ff service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:24:53.605515 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd718c5b-8685-411d-b1cb-9c699e193abf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5177fa4d168c4c219e09c7178bbbc790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:24:53.606148 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd718c5b-8685-411d-b1cb-9c699e193abf service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:24:53.606461 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd718c5b-8685-411d-b1cb-9c699e193abf service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:24:53.606768 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd718c5b-8685-411d-b1cb-9c699e193abf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:24:53.609115 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fd718c5b-8685-411d-b1cb-9c699e193abf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:24:53.621378 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd718c5b-8685-411d-b1cb-9c699e193abf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:24:53.623179 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd718c5b-8685-411d-b1cb-9c699e193abf service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:24:53.633383 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1500/2995] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:24:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:24:53.715585 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a93b1-ff07-4466-a2aa-796d0474c10a service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:24:53.726642 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a93b1-ff07-4466-a2aa-796d0474c10a service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:24:53.732690 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da961ca-8b04-4e0f-9ca1-293664a8f562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:24:53.733350 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da961ca-8b04-4e0f-9ca1-293664a8f562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:24:53.733633 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da961ca-8b04-4e0f-9ca1-293664a8f562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:24:53.733904 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da961ca-8b04-4e0f-9ca1-293664a8f562 admin admin] PATH_INFO: `/v3/users/a29bbe80a3cc44a58ddcd0dd784f11bf` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:24:53.735169 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da961ca-8b04-4e0f-9ca1-293664a8f562 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a29bbe80a3cc44a58ddcd0dd784f11bf)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:24:53.737111 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da961ca-8b04-4e0f-9ca1-293664a8f562 admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:24:53.940228 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4da961ca-8b04-4e0f-9ca1-293664a8f562 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a29bbe80a3cc44a58ddcd0dd784f11bf'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:24:54.052103 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4da961ca-8b04-4e0f-9ca1-293664a8f562 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a29bbe80a3cc44a58ddcd0dd784f11bf'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:24:54.052731 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4da961ca-8b04-4e0f-9ca1-293664a8f562 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a29bbe80a3cc44a58ddcd0dd784f11bf'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:24:54.053337 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4da961ca-8b04-4e0f-9ca1-293664a8f562 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a29bbe80a3cc44a58ddcd0dd784f11bf'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:24:54.065999 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-4da961ca-8b04-4e0f-9ca1-293664a8f562 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a29bbe80a3cc44a58ddcd0dd784f11bf'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:24:54.079412 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1496/2996] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 10:24:53 2022] DELETE /identity/v3/users/a29bbe80a3cc44a58ddcd0dd784f11bf => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:24:54.328806 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd718c5b-8685-411d-b1cb-9c699e193abf service placement] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:24:54.340930 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd718c5b-8685-411d-b1cb-9c699e193abf service placement] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:24:54.400502 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b67b96-c943-48fb-8e03-ae414b9db0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:24:54.401134 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b67b96-c943-48fb-8e03-ae414b9db0aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:24:54.401417 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b67b96-c943-48fb-8e03-ae414b9db0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:24:54.401692 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b67b96-c943-48fb-8e03-ae414b9db0aa admin admin] PATH_INFO: `/v3/projects/bfd00fdbe14e451fb3634ef9077b3a76` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:24:54.402836 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b67b96-c943-48fb-8e03-ae414b9db0aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfd00fdbe14e451fb3634ef9077b3a76)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:24:54.404728 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b67b96-c943-48fb-8e03-ae414b9db0aa admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:24:54.511489 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e5b67b96-c943-48fb-8e03-ae414b9db0aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfd00fdbe14e451fb3634ef9077b3a76', 'request_id': 'req-e5b67b96-c943-48fb-8e03-ae414b9db0aa'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:24:54.589333 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e5b67b96-c943-48fb-8e03-ae414b9db0aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfd00fdbe14e451fb3634ef9077b3a76', 'request_id': 'req-e5b67b96-c943-48fb-8e03-ae414b9db0aa'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:24:54.595715 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e5b67b96-c943-48fb-8e03-ae414b9db0aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfd00fdbe14e451fb3634ef9077b3a76', 'request_id': 'req-e5b67b96-c943-48fb-8e03-ae414b9db0aa'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:24:54.608519 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e5b67b96-c943-48fb-8e03-ae414b9db0aa admin admin] The token cache is being invalidate because project bfd00fdbe14e451fb3634ef9077b3a76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 10:24:54.608830 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e5b67b96-c943-48fb-8e03-ae414b9db0aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:24:54.609515 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e5b67b96-c943-48fb-8e03-ae414b9db0aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:24:54.611121 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1501/2997] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 10:24:54 2022] DELETE /identity/v3/projects/bfd00fdbe14e451fb3634ef9077b3a76 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:24:54.615323 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da961ca-8b04-4e0f-9ca1-293664a8f562 admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:24:54.616860 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-4da961ca-8b04-4e0f-9ca1-293664a8f562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:24:54.631204 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da961ca-8b04-4e0f-9ca1-293664a8f562 admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:24:54.698230 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36cddc3-3434-490b-ab6d-83565542f4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:24:54.704277 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36cddc3-3434-490b-ab6d-83565542f4cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:24:54.704277 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36cddc3-3434-490b-ab6d-83565542f4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:24:54.704277 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36cddc3-3434-490b-ab6d-83565542f4cb admin admin] PATH_INFO: `/v3/users/5fa378fcb82c492aa01013b3a2c7edbd` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:24:54.704277 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36cddc3-3434-490b-ab6d-83565542f4cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fa378fcb82c492aa01013b3a2c7edbd)` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:24:54.704277 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36cddc3-3434-490b-ab6d-83565542f4cb admin admin] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:24:54.974980 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b36cddc3-3434-490b-ab6d-83565542f4cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fa378fcb82c492aa01013b3a2c7edbd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:24:55.029081 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b36cddc3-3434-490b-ab6d-83565542f4cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fa378fcb82c492aa01013b3a2c7edbd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:24:55.029713 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b36cddc3-3434-490b-ab6d-83565542f4cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fa378fcb82c492aa01013b3a2c7edbd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:24:55.030319 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b36cddc3-3434-490b-ab6d-83565542f4cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fa378fcb82c492aa01013b3a2c7edbd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:24:55.036294 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:24:55.043276 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.notifications [None req-b36cddc3-3434-490b-ab6d-83565542f4cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fa378fcb82c492aa01013b3a2c7edbd'} {{(pid=81611) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:24:55.056439 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1497/2998] 10.222.0.57 () {58 vars in 1224 bytes} [Fri Sep 2 10:24:54 2022] DELETE /identity/v3/users/5fa378fcb82c492aa01013b3a2c7edbd => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:24:55.322635 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b67b96-c943-48fb-8e03-ae414b9db0aa admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:24:55.324151 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e5b67b96-c943-48fb-8e03-ae414b9db0aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:24:55.337123 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b67b96-c943-48fb-8e03-ae414b9db0aa admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:24:55.345218 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:24:55.394474 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d93308-25d7-4f22-8131-a6ed294cc7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cffe229f2f843e394c2248b000ecb37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae5820815add48cd913951058cc9251c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:24:55.395049 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d93308-25d7-4f22-8131-a6ed294cc7b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:24:55.395324 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d93308-25d7-4f22-8131-a6ed294cc7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:24:55.395594 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d93308-25d7-4f22-8131-a6ed294cc7b4 admin admin] PATH_INFO: `/v3/projects/1a25144970a84b8eb90df62ab2232985` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:24:55.396697 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d93308-25d7-4f22-8131-a6ed294cc7b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a25144970a84b8eb90df62ab2232985)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:24:55.398573 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d93308-25d7-4f22-8131-a6ed294cc7b4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:24:55.532639 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-79d93308-25d7-4f22-8131-a6ed294cc7b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a25144970a84b8eb90df62ab2232985', 'request_id': 'req-79d93308-25d7-4f22-8131-a6ed294cc7b4'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:24:55.623648 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-79d93308-25d7-4f22-8131-a6ed294cc7b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a25144970a84b8eb90df62ab2232985', 'request_id': 'req-79d93308-25d7-4f22-8131-a6ed294cc7b4'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:24:55.629786 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-79d93308-25d7-4f22-8131-a6ed294cc7b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a25144970a84b8eb90df62ab2232985', 'request_id': 'req-79d93308-25d7-4f22-8131-a6ed294cc7b4'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:24:55.641367 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-79d93308-25d7-4f22-8131-a6ed294cc7b4 admin admin] The token cache is being invalidate because project 1a25144970a84b8eb90df62ab2232985 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 02 10:24:55.641656 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-79d93308-25d7-4f22-8131-a6ed294cc7b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:24:55.642323 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-79d93308-25d7-4f22-8131-a6ed294cc7b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 02 10:24:55.643879 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1502/2999] 10.222.0.57 () {58 vars in 1233 bytes} [Fri Sep 2 10:24:55 2022] DELETE /identity/v3/projects/1a25144970a84b8eb90df62ab2232985 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 02 10:25:05.194808 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=9,keystone:DELETE=1 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:25:05.638077 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:27:21.130553 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36cddc3-3434-490b-ab6d-83565542f4cb admin admin] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:27:21.132049 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-b36cddc3-3434-490b-ab6d-83565542f4cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:27:21.234247 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36cddc3-3434-490b-ab6d-83565542f4cb admin admin] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:27:21.293255 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b9504-3e90-48a4-9713-e50a0fbac8d0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5177fa4d168c4c219e09c7178bbbc790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:27:21.294262 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b9504-3e90-48a4-9713-e50a0fbac8d0 service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:27:21.294805 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b9504-3e90-48a4-9713-e50a0fbac8d0 service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:27:21.295184 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b9504-3e90-48a4-9713-e50a0fbac8d0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:27:21.297076 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f26b9504-3e90-48a4-9713-e50a0fbac8d0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:27:21.309067 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26b9504-3e90-48a4-9713-e50a0fbac8d0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:27:21.313692 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26b9504-3e90-48a4-9713-e50a0fbac8d0 service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:27:21.320866 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1498/3000] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:27:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:27:31.316386 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=73 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:29:24.090247 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d52e244-a5d4-4a52-b3aa-8f272fd5de27 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:29:24.090869 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d52e244-a5d4-4a52-b3aa-8f272fd5de27 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:29:24.090869 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d52e244-a5d4-4a52-b3aa-8f272fd5de27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:29:24.140481 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5d52e244-a5d4-4a52-b3aa-8f272fd5de27 None None] MFA Rules not processed for user `5177fa4d168c4c219e09c7178bbbc790`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 02 10:29:24.141608 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5d52e244-a5d4-4a52-b3aa-8f272fd5de27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:29:24.150115 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1503/3001] 10.222.0.16 () {58 vars in 1015 bytes} [Fri Sep 2 10:29:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 02 10:29:24.163979 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b9504-3e90-48a4-9713-e50a0fbac8d0 service placement] Authenticating user token {{(pid=81611) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 02 10:29:24.165349 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.fernet_utils [None req-f26b9504-3e90-48a4-9713-e50a0fbac8d0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81611) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 02 10:29:24.178053 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b9504-3e90-48a4-9713-e50a0fbac8d0 service placement] Validating token access rules against request {{(pid=81611) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 02 10:29:24.184531 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1ae9d-ee08-4462-9e76-17295a0d4812 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5177fa4d168c4c219e09c7178bbbc790', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd18a5117a03b4f21ab3290a15680de66', 'project_domain_id': 'default', 'roles': ['member', 'service', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81611) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 02 10:29:24.185142 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1ae9d-ee08-4462-9e76-17295a0d4812 service placement] REQUEST_METHOD: `GET` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 02 10:29:24.185420 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1ae9d-ee08-4462-9e76-17295a0d4812 service placement] SCRIPT_NAME: `/identity` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 02 10:29:24.185692 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1ae9d-ee08-4462-9e76-17295a0d4812 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81611) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 02 10:29:24.194440 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d1ae9d-ee08-4462-9e76-17295a0d4812 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 02 10:29:24.196071 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d1ae9d-ee08-4462-9e76-17295a0d4812 service placement] RBAC: Authorization granted {{(pid=81611) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 02 10:29:24.207408 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: [pid: 81611|app: 0|req: 1499/3002] 10.222.0.57 () {60 vars in 1365 bytes} [Fri Sep 2 10:29:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 02 10:29:34.138086 dv-cinder-850831-3-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=13 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 02 10:29:34.202918 dv-cinder-850831-3-iscsi devstack@keystone.service[81611]: DEBUG dbcounter [-] [81611] Writing DB stats keystone:SELECT=6 {{(pid=81611) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}